Automate dynamic web tables
A table is a kind of HTML data displayed with the help of the <table> tag in conjunction with the <tr> and <td> tags. Although there are other tags for creating tables, these are the basics for creating a table in HTML:
<tr>defines a row.<td>specifies a column cell of a table. Each cell in the Excel sheet can be represented as<td>in the HTML table. The<td>elements are the data containers that can enclose all sorts of HTML elements like text, images, lists, other tables, and so on.
An Excel sheet is a simple example of table structures. Whenever you put some data to Excel, you add some heading as well. In HTML, the <th> tag is used for headings.
<table border="1" width="100%">
<tbody>
<tr>
<td>Automation Tool</td>
<td>Licensing</td>
<td>Notes</td>
</tr>
<tr>
<td>RPA Express</td>
<td>Free</td>
<td>Can be extended with multiple bots</td>
</tr>
<tr>
<td>WorkFusion SPA</td>
<td>Commercial</td>
<td>Contains Cognitive component</td>
</tr>
</tbody>
</table>
The <th> tag stands for a table cell heading and is used instead of <td> when the cell content is a heading instead of the actual cell data.
It is the obvious choice inside the <thead> element that can contain, for example, the first row of your table, but you can also use it for the first column to indicate table row headings.
The <tfoot> table footer is always displayed under the <tbody>, even if its code is before the table body. A table can have a name given using the <caption> tag.
<table border="1" width="100%">
<caption>Sample Table</caption>
<thead>
<tr>
<th>Automation Tool</th>
<th>Licensing</th>
<th>Notes</th>
</tr>
</thead>
<tfoot>
<tr>
<td colspan="3"><em>Public Info</em></td>
</tr>
</tfoot>
<tbody>
<tr>
<td>RPA Express</td>
<td>Free</td>
<td>Can be extended with multiple bots</td>
</tr>
<tr>
<td>WorkFusion SPA</td>
<td>Commercial</td>
<td>Contains Cognitive component</td>
</tr>
</tbody>
</table>
To handle dynamic web tables, first, inspect the table cell and get its HTML location. In most cases, tables contain text data, and you can extract the data given in each row or column of the table.
Sometimes, tables have links or images. You can perform any action on those elements if you find the HTML location of the containing cell. For a sample, refer to the automation practice table page.
Example 1
Let’s take the above table and choose Row 2 Column 3 cell (Dubai):
//*[@id="content"]/table/tbody/tr[1]/td[2]
If you divide the XPath into three different parts, you have:
- Table location on the web page:
//*[@id="content"]/. - Table body (data):
table/tbody/. - Table row 1 and table column 2 (the first visible row is in
<thead>, and the first visible column is<th>):tr[1]/td[2]
If you use this XPath, you get the specified table cell. To get the "Selenium" text from the table cell, use the getText() method of the WebDriver element:
open('https://rpa-tutorial.s3.amazonaws.com/trainings/table-automation.html')
def cellText = $(byXpath("//*[@id='content']/table/tbody/tr[1]/td[2]")).getText()
Example 2
Tables can contain a large amount of data, and you may need to pass rows and columns dynamically.
In that case, build your XPath using variables:
open('https://rpa-tutorial.s3.amazonaws.com/trainings/table-automation.html')
def sRow = 1
def sCol = 2
def cellText = $(byXpath("//*[@id='content']/table/tbody/tr[${sRow}]/td[${sCol}]")).getText()
Example 3
If the row and columns are dynamic, and all you know is the Text value of any cell, take out the corresponding values of that particular cell.
For example, you have to record all possible values in the "Licensing" column from the above example:
open('https://rpa-tutorial.s3.amazonaws.com/trainings/table-automation.html')
def colHeading = 'Licensing'
def columnValues = []
def tBodyXpath = "//*[@id='main']/div[2]/div/div[2]/table/tbody"
def totalRows = $$(byXpath("${tBodyXpath}/tr")).size()
for(int i=1;i<=3;i++) {
def sValue = $(byXpath("${tBodyXpath}/tr[1]/th[${i}]")).getText()
if(sValue.equalsIgnoreCase(colHeading)) {
// If the sValue match with the description, it will initiate one more inner loop for all the columns of 'i' row
for(int j=2; j<=totalRows; j++){
def tempVal = $(byXpath("${tBodyXpath}/tr[${j}]/td[${i}]")).getText()
columnValues.add(tempVal)
}
break
}
}
log.warn(columnValues.toString())
Example 4
Click the Detail link of the first row and the last column.
open('https://rpa-tutorial.s3.amazonaws.com/trainings/table-automation.html')
$(byXpath("//*[@id='content']/table/tbody/tr[1]/td[6]/a")).click()
Example 5
Get the value from the Dubai cell using a dynamic XPath.
Print all the column values of the Clock Tower Hotel row.
open('https://rpa-tutorial.s3.amazonaws.com/trainings/table-automation.html')
def sRow = 1
def sCol = 2
def sColumnValue = []
//Here we are locating the xpath by passing variables in the xpath
def sCellValue = $(byXpath(".//*[@id='content']/table/tbody/tr[" + sRow + "]/td[" + sCol + "]")).getText()
println(sCellValue)
def sRowValue = 'Clock Tower Hotel'
//First loop will find the 'ClOCK TOWER HOTEL' in the first column
for (int i=1; i<=5; i++) {
def sValue = null
sValue = $(byXpath(".//*[@id='content']/table/tbody/tr[" + i + "]/th")).getText()
if(sValue.equalsIgnoreCase(sRowValue)){
// If the sValue match with the description, it will initiate one more inner loop for all the columns of 'i' row
for (int j=1; j<=5; j++){
def temp = $(byXpath(".//*[@id='content']/table/tbody/tr[" + i + "]/td["+ j +"]")).getText()
sColumnValue.add(temp)
}
break
}
}
log.warn(sColumnValue.toString())
Example 6
Get the HTML Table Example table.
Put each table row into Export as a separate column (variable).
<?xml version="1.0" encoding="UTF-8"?>
<config xmlns="http://web-harvest.sourceforge.net/schema/1.0/config" scriptlang="groovy">
<robotics-flow>
<!-- Internet explorer driver -->
<robot name="webDriver" driver="internet explorer" close-on-completion="true" start-in-private="true">
<script><![CDATA[
timeouts(15 * 1000)
open("http://www.w3schools.com/html/html_tables.asp")
pageSource = driver().getPageSource()
]]></script>
</robot>
</robotics-flow>
<var-def name="xmlSource">
<html-to-xml>
<script return="pageSource" />
</html-to-xml>
</var-def>
<var-def name="tableXml">
<xpath expression="//table[@id='customers']">
<script return="xmlSource" />
</xpath>
</var-def>
<script><![CDATA[
rpaVariables = new HashMap()
rpaVariables.put("source", xmlSource.toString())
rpaVariables.put("tableXml", tableXml.toString())
]]></script>
<loop item="rowXml" index="idx">
<list>
<xpath expression="//tr">
<script return="tableXml" />
</xpath>
</list>
<body>
<script><![CDATA[
values = new ArrayList()
]]></script>
<loop item="valString">
<list>
<xpath expression="//td/text()">
<var name="rowXml" />
</xpath>
</list>
<body>
<script><![CDATA[
values.add(valString.toString())
]]></script>
</body>
</loop>
<script><![CDATA[
rpaVariables.put("column_" + idx.toString(), values.toString())
]]></script>
</body>
</loop>
<!-- Export values to the output CSV file -->
<export include-original-data="true">
<loop item="rpaVar">
<list>
<script return="keys">
keys = new ArrayList(rpaVariables.keySet())
</script>
</list>
<body>
<single-column name="${rpaVar}">
<template>${rpaVariables.get(rpaVar.toString())}</template>
</single-column>
</body>
</loop>
</export>
</config>