Inspector

Inspector is a tool to explore properties and values of objects in native Windows or multiple frameworks, Java, and SAP applications, which you automate with your recording, providing functionality for quick element selector search.

You can extract selectors with the help of specific tools, one of which is Inspector as a component of Work.AI Developer. The usual workflow contains the following steps: Inspect UI controls > Check selectors > Apply selectors.

To perform actions in any desktop application, a bot has to find particular elements to interact with (type, click, focus, and so on). For that purpose, a selector, or a unique link to a window or UI control, is used. For a description of selectors, see Use inspecting tools for desktop automation | General overview.

Launch Inspector

To open Inspector, right-click the Work.AI Developer icon in the system tray and select Inspector.

The Inspector window contains the following components:

1. Browser contains all the currently running applications or the list of UI controls when exploring the selected window.

2. Result List displays the list of all properties related to the selected window or UI control.

3. Selector is the field to hold a unique selector.

4. Type allows switching between selector types. When you select an item in the Type filter, the selector string is rebuilt to match the given notation. There are three selector types you can choose from:

   

There are three selector types you can choose from:

• CSS, for example, .Button[name="Maximize"]
• Object, for example, [CLASS:Button; NAME:Maximize]
• XPath, for example, //*[@class='Button'][@name='Maximize']

Inspect UI controls

You can inspect UI controls in different ways.

• Сapture (inspect) controls by hovering over them.
• Select controls in the Browser tree as in developer tools.
• Check how UI controls work in applications by setting commands.
• Test if controls match given selectors by using the search function.

To start exploring application components, select the application you want to inspect in Browser .

tip

If you start an application after launching Inspector, click Refresh Browser to update the list.

If the system or other user runs a process, you cannot inspect it. Then, you are notified about a necessity either to run Inspector as Administrator or disable the user account control.

note

Inspection of SAP GUI desktop client differs from the usual inspecting workflow. For more information, refer to SAP GUI desktop automation.

Capture

Inspector provides the Capture option enabling you to enter the inspection mode and explore the properties of application components.

1. Click Capture.

   

2. Hover the cursor over a window control. It gets highlighted directly in the application.

   

3. Click the desired window control to select it. The target control opens in the Inspector window automatically, with its properties displayed in Result List and a selector value available in the Selector field.

   

There are cases when you need to click a control without selecting it, for example, to explore other application options or switch to another application. Use the black menu as a prompt.

• To click the control without selecting, use the left mouse click + Ctrl.
• To cancel capturing and exit the inspection mode, press Esc.

Explore in Browser

If it is hard to get the right selector by capturing, for example, a selector is hidden in the application UI, or you do not want to click through the application, you can switch between controls directly in the Browser area.

1. Double-click the application or process in Browser, or click Show UI Components in Result List.

   

2. Navigate in Browser using the mouse or keyboard arrows.

   

   The selected component gets highlighted in the application window. Its properties are available in Result List and the corresponding selector in the Selector field.

3. To exit the inspection mode, click Back to Applications and Windows in Browser.

Test windows and UI controls

Apart from highlighting, you can send different actions to any desired application to emulate the bot's behavior and make sure it functions as expected.

To check that the bot can interact with the application and its components (for example, Notepad), do as follows:

1. To test interaction with the application, select the application in Browser, right-click, and select one of the commands, for example, Type ("hello"). The command is performed in the application window.

   

2. To test if the application components respond, double-click the application in Browser or click Show UI Components in Result List. Right-click any component in Browser and select one of the commands, for example, Click. The command is performed for the selected component.

   

tip

To view or hide applications and elements within them, click Expand/Collapse in front of the application or process you are interested in. You can also click Expand All and Collapse All in the Browser options.

The set of actions applied to applications differs from those applied to UI controls.

Window commands	UI control commands
Activate Type Press keys Maximize or minimize Close or kill	Focus Move mouse Click, double-click, or right-click Type or set text Press keys

Inspect windows

You can also create custom window selectors and assess them against all open windows. On the Applications and Windows screen, select a needed window to get its selector. The inspecting procedure is the same as for UI controls.

Test selectors

Before putting a selector to a Bot Task configuration, test if it works correctly. To do that, search for the UI control matching the given selector.

1. Copy your selector into the Selector field and press Enter.

   

2. If the selector exists, Inspector finds the corresponding window control and selects it in Browser with detailed information in Result List. The target control is highlighted in the application.

   

Inspector checks for selector uniqueness and supports collections of selectors corresponding to the search criteria. If there is more than one selector with the same attributes in the search results, switch between selectors by clicking the Back and Forward arrows until you find the needed selector.

tip

Open the application a selector belongs to, as the search is performed among UI controls of the selected application. Otherwise, the selector will not match any components. If you still cannot find your selector, make sure you copied it correctly. The Instance and Text attributes can also facilitate the search.

Apply selectors

As soon as the target component is found and tested, you can copy it for further usage in Recorder or Eclipse.

Copy to Recorder (UI Control)

1. Press Ctrl + A to select your unique object selector.

2. Press Ctrl + C to copy.

   

3. Open your recording in Recorder select an action from Actions Library and add it to Actions Flow.

4. In the action properties window, select Target > Window control > Selector and press Ctrl + V. The unique object selector value can also be used for Selector of the Mouse Click action to assign to a specific control in the application.

   

Copy to Eclipse

To copy a selector to a Bot Task configuration, open Eclipse and paste your selector into the code.

Get value from field in Windows application

There are two ways of getting a value from a window control.

Before using selectors in a recording, switch to the target application with the help of the Window action.

Way 1

1. Create a Recorder variable of type String.

2. Open Inspector and find a selector of the necessary field in the application.

3. Apply the Mouse Click action > Window control.

   

4. Use the Enter Keystrokes (Ctrl + C) action to copy the value from the field. If needed, use Enter Keystrokes (Ctrl + A) beforehand.

5. Use the Clipboard action to set the copied value into your variable.

Way 2

1. Create a Recorder variable of type String.

2. Open Inspector and find a selector of the necessary field in the application.

3. To get the text from the field, use the Script action with the code as in the example below.

   

Special cases

Inspect Office 365 applications

Pay special attention when retrieving selectors of UI controls related to Office 365 applications. After Microsoft software updates, attribute values may change, and selectors previously used in your scripts will not be unique or stop working.

Excel cell selectors differ in various versions of Microsoft Office 365:

• A previous version of Microsoft Excel: [CLASS:DataItem; NAME:A1]

• A newer version of Microsoft Excel: [CLASS:DataItem; NAME:"A" 1]

  

warning

Be careful with the selectors related to UI controls of Office 365 applications, as Microsoft software updates can change selector values. It is recommended to reinspect UI controls or use Recorder for Excel and other Office 365 related actions.

Focus on hidden elements

If a UI control is collapsed and cannot be visible on the application screen, you can easily expand it to get a selector. It is especially useful when inspecting complex trees or tabs. To open a collapsible UI control, do as follows.

1. Right-click any hidden element within the tree and select Focus.

   

2. When you expand the element, copy its selector to use in your action flow.

   

You can also activate a hidden element using the focus() method in your script: $('tree_hidden_element_selector').focus()

Export to XML

You can use XML export features to build selectors. You can export all windows or specific component trees.

Export windows

1. To export all the windows, click Export Windows to XML in Browser.

   

2. View the XML code on the Source tab.

   

Export component tree

You can get the whole UI component tree and manipulate it in your text editor or IDE.

1. Select the application or process in Browser.

2. Double-click it, or click Show UI Components.

3. Click Export XML.

4. View the XML code on the Source tab.

   

Troubleshooting

Wrong Java version

If you cannot set up RPA Package Java as the default Java for IE (jdk/jre.exe installer can do that anyway), load Java with the build version not higher than the RPA Package one.

Having RPA Package Java version 8u92 and IE pointed to 8u101, Inspector cannot connect to 8u101 JVM and display elements selectors. Even when a selector is found through a different Machine, RPA Agent cannot find it.

Installing 8u91 and pointing to IE fixes that issue. Thus, Inspector starts working, and selectors are successfully executed.

No element found

Sometimes, after Windows security updates have been installed, your RPA agent stops finding elements, and Inspector doesn't detect any jJva processes for inspection.

In the command window, instead of:

...inspector.controller.JvmInspectorControllerImpl.updateVirtualMachineTree[:73] - Updating JVM Tree...
...common.utils.VirtualMachineConnector$VMConnection.connect[:140] - Using jvmAgentPath =     c:\rpa\rpa-grid\dependency\rpa-jvmagent.jar     
...inspector.controller.JvmInspectorControllerImpl.connectToJvmAndUpdateVmTree[:236] - Successfully connected     to: 23652

There are only updating JVM tree attempts:

bad
...inspector.controller.JvmInspectorControllerImpl.updateVirtualMachineTree[:73] - Updating JVM Tree...
...inspector.controller.JvmInspectorControllerImpl.updateVirtualMachineTree[:73] - Updating JVM Tree...
...inspector.controller.JvmInspectorControllerImpl.updateVirtualMachineTree[:73] - Updating JVM Tree...

You can start troubleshooting by looking at C:/rpa/java/bin/jconsole.exe.

When JConsole detects external processes but can't get any information, the warning message is as follows: "The management agent is not enabled on this process". That means JConsole cannot access PID files created by a Java process.

To resolve the issue, follow the instruction:

1. Close all Java programs.
2. Check where the %TEMP% and %TMP% environment variables are pointed.
3. Clean your TEMP folder, most likely in C:/Users/{username}/AppData/Local/Temp.
4. Run settings-rdp.reg from the RPA setup folder.
5. Re-log in into your PC.
6. Run all the Java programs, test with JConsole, and then use Inspector.
7. Check if C:/Users/{username}/AppData/Local/Temp/hsperfdata_{username} has files corresponding to Java process IDs.