Troubleshoot AI Agent issues

Blocking and non-blocking exception handling

In Work.AI, if an error occurs during the execution of a Business Process (BP) step on one record, it blocks the execution of all other records of this BP. This behavior is not good for automating business-critical workflows that have to run unattended. Still, it's simpler to implement and, sometimes, good enough to let the automation deliver value. The ODF 2-powered automation can handle exceptions in blocking and non-blocking ways.

The default behavior of ODF 2 is to function in accordance with the FailFastErrorHandling strategy that doesn't perform any handling, so the default platform behavior takes place. The ODF 2 Example project provides a more advanced error handling strategy. The strategy is to record all unhandled exceptions to a special table in Data Stores and mark the corresponding Transaction to reflect that. Therefore, if a Transaction makes it to the end of a Business Process being ignored by all Bot Tasks, this does not mean it is successful—you have to check its HAS_ERRORS attribute.

To override the current behavior, in the usecase.properties file, modify the errorHandlingModule property to your custom Feather module. For more details, refer to Exception handling.

Errors when running BP in Control Tower

info

For exceptions in a BP, it is important to distinguish between exceptions of your custom code, ODF 2 framework exceptions, and those outside of the ODF 2 framework. It is recommended not to use the FailFastErrorHandling strategy even if you are fine with the exception handling strategy like "do nothing" and introduce your own exception hierarchy.

Error in Event log for BP run

If an error occurs in the Control Tower error log, it means either of the two things:

• The current exception handling strategy assumes blocking the execution. In this case, act as you usually do to handle stack traces.

• Something happens outside of the ODF 2 code. Analyze the stack trace in the Control Tower error log. Review the RPA Worker log. Most likely, it is an infrastructure-related issue.

To access the Control Tower and RPA Worker logs via Work.AI Developer, use the Launcher application. To access logs via the Work.AI platform, use Platform Monitor as described in the following guides:

• Collect and view logs
• Tasks execution analysis

Warning in Event log for BP run

When ODF 2 catches an otherwise unhandled exception, and the error handling strategy is non-blocking, the latter logs a warning that states the following: "Error was raised inside ODF task”.

Check the ds_uc_<uc_name>_exception_v<version> table for a complete stack trace:

• OdfFrameworkException is thrown by ODF 2 itself. If your custom code never throws it, this can be an internal framework error.

In any case, act the same way as you usually do to handle stack traces.

Encountered issues

Project creation issues

If creating a project from an Archetype fails, make sure the AI Agent name and code are specified because these are required parameters. For more information, refer to the Start Maven project article.

Database issues

Incorrect table structure

Errors due to the incorrect table structure (for instance, expected columns missing) are often associated with Liquibase migrations. Either a change in an entity class was not reflected in migrations, or migrations were not appropriately applied to the database. For details, refer to Data Model Management | Use Liquibase migrations.

BP errors related to the database structure are recorded in the Event log as: Error was raised inside ODF task. To check the error cause, search the Worker log for the exception stack trace with detailed information.

Incorrect table name

Cases, when ODF 2 expects a table to be named not like it is really named, are usually due to a misconfiguration. The structure of the Data Store table name that ODF 2 expects to find is as follows:

ds_uc_<usecase-code>_<table-name>_v<version>

The AI Agent code is defined in the usecase.properties file. For projects created from an Archetype, it is mainly specified in the <properties> section of pom.xml and is added to the above file via templating in Maven.

To resolve the table name-related issues, explore the following possible root causes:

• If a part of a table name differs from what ODF 2 expects, it can mean the AI Agent code was changed, but migrations were not updated accordingly.

• If ODF 2 attempts to find a table named like ds_uc_<table-name>_v<version>, the behavior implies the AI Agent code is missing from the usecase.properties file. This can also mean that there is no usecase.properties file in the classpath of the BCB (Bot Config Bundle) containing the Task class from which the associated Bot Task was generated. If it is a new module in a project, make sure you added it in accordance with the Add BCB module article.

• A table name is derived from the name of an entity class. If the table name differs from the one stored in the database, it can mean that the associated entity class was renamed, which was not reflected in migrations, or migrations were not applied to the database properly. For details, refer to Data Model Management | Use Liquibase migrations.

Similar to the AI Agent code, the AI Agent version is defined in the usecase.properties file. Therefore, for resolving issues related to the AI Agent version misconfigurations, use the same approach.

Deadlocks on tables with large amount of data

Entities extended from the com.workfusion.odf2.core.orm.OdfEntity class have a field with the type UUID as a unique identifier. This field is also used as a search key to find a particular object.

In OrmLite, the fields with the type UUID are mapped to columns with the varchar data type (varchar(36) in the case of ODF 2 fields). Due to the bug in the MSSQL jdbs driver, varchar(36) in a Java prepared statement is transformed to varchar(4000) in an MSSQL request on the server.

For tables with a large amount of data (more than 100 thousand records), such transformation, along with using other varchar fields as search keys, dramatically decreases the performance of select requests and can cause a deadlock on updates.

To avoid deadlocks, you can use the raw query functionality of OrmLite. Instead of calling an update using a repository class, use constructions like the one below:

com.j256.ormlite.dao.Dao.updateRaw(”update entity set some_file=? where ds_uc_entity_v1='uuid_value', newValue)

Asset Bundle import issues

See Asset Bundle troubleshooting for the troubleshooting guide.

Versioning issues

When building a project, if you increase its version only, some assets in the project may have their versions unchanged, which can result in a number of problems during deployment. Make sure your asset naming conventions are in accordance with the versioning plugin documentation, and you are following the versioning flow described in it. In the documentation, you can also find potential problems and possible solutions.

Double version in Data Store names issue

When experimenting with AI Agent and Data Store versions, you may try to manually change [Project]-package/src/main/resources/meta-info.json. It is not recommended as the file's content is dynamically changed. During the build, versions from the root pom.xml are copied into meta-info.json. Thus, the only place where versions should be adjusted manually is the root pom.xml file.

The cause of a double-versioning suffix, for example, uc_test_monitor_v2_v3, is as follows. The plugin uses meta-info.json and takes the old data model version to find a suffix for replacement. As a result, if you manually change the data model version both in pom.xml and meta-info.json, the plugin cannot find the original (old) suffix, and an additional version is added to the suffix.

Issues related to Bot Task jUnit tests

The most common failures of Bot Task tests are due to no mock infrastructure created as required by the task code.

In this case, the first suspect is database tables. The Bot Task testing engine cannot create any tables by default—you have to create even the transaction table explicitly before running a task. If a required table is missing, you will know it by the error message of similar content: org.h2.jdbc.JdbcSQLSyntaxErrorException: Table "ds_uc_odf2_it_transaction_v1_0" not found.

Typically, tables for Transaction, ErrorEntity, MonitorEntity, and MonitorConfigurationEntity are required. To mock those, inject com.workfusion.odf2.junit.OrmSupport and use its API as described in the Bot Task JUnit documentation.

Other standard services a tested task can require and you can mock are S3, OCR, and Secrets Vault. So, if an error message or a stack trace contains references to the services, make sure they are properly mocked in your test.

Issues related to Spoke tests

HTTP 401

When running the Control Tower environment for the first time with SSO enabled, you can fail to authenticate under an existing valid user if there is a known CT-to-Keycloak synchronization issue. After the first login to the CT web interface, the issue is resolved naturally—without any special troubleshooting efforts.

The second known issue is related to the Spoke design in terms of executing HTTP requests. If authentication fails for some reason (for instance, a user has entered a wrong password by mistake) during a test, Spoke does not stop the execution immediately. Instead, it calls a REST method—usually, /bundle-import—without a valid token from the server. At this point, the server responds with the 403 error—meaning the incoming request includes no authentication token. After that, if the retry mechanism is enabled, Spoke performs the same operations, including authentication, repeatedly with the same result. After five failed attempts, Keycloak locks the user temporarily, and all subsequent requests fail, even with valid credentials.

So, when a user specifies wrong credentials, Spoke won't let you know that. Instead, it sends multiple authentication requests, blocking the user completely. After that, the only way to recover is to wait until the user is unlocked automatically or goes to the Keycloak UI to unlock manually.

HTTP 403

Such an error shows that the user, under which Spoke connects to Control Tower, lacks the permissions required to upload an Asset Bundle.

Timeout exceeded

When importing an Asset Bundle to Control Tower via Spoke, you can get the following message: “Upload failed: wait time elapsed.” It means that the bundle upload is taking more time than the Asset Bundle import API is configured to wait. When you call controlTower.importAssetBundle(myBundle), the API uses the default five-minute timeout. There is an option to pass a custom import timeout by calling controlTower.importAssetBundle(myBundle, Await.atMost(myDuration)).

If such an error appears suddenly, it can mean the load of the Control Tower instance is unusually heavy, or network conditions are degraded abruptly.

The same reasons can cause a test to fail on a subsequent stage as you wait for a BP execution to finish. In this case, the timeout is configured in the .run().waitFor(myDuration) clause.

info

By inherent design, some BPs cannot be completed without manual interactions or depend on external services with complex or unpredictable behaviors, and therefore the BPs consistently cause exceeded timeouts when run. Such BPs are not suitable for running via Spoke without modifications.

To build integration tests for the BPs, mock the tasks causing the undesired behavior, which can also include Monitor steps. For more information on mocking Bot Tasks in Spoke, refer to the Business Process integration testing article.

Other Asset Bundle import problems

As Spoke uses the Asset Bundle import API, the issues described in the previous section also apply to Spoke-based tests.

Dependency management issues

info

It is highly recommended to read the official documentation on the Maven dependency management mechanism.

When Maven builds a project, it needs to understand what dependency version to use. Usually, any project explicitly specifies a version for every dependency it uses. But when it comes to transitive dependencies (which means dependencies of a dependency), ambiguity may arise.

For example, let's assume that some project uses A and B dependencies, both with clearly defined versions. In turn, both A and B dependencies use C dependency, but A needs C version 1, and B needs C version 2. When Maven tries to resolve a project, it decides what version of C must be used—C:1 or C:2. The actual decision depends on many factors. Usually, the newest version is used, but it is not guaranteed. More importantly, it is not guaranteed on a build-to-build basis, especially if builds are done in different environments with different Maven versions. If a C:2 is chosen, and there are some backwards-incompatible changes in it comparing to C:1, A may encounter some weird behaviour or even break on runtime, usually complaining about some missing method, wrong method signature, or some class being not what it is expected to be. If C:1 is chosen, and B uses some features not present in C:1, the same can happen with B.

So, if a project fails on runtime with some sort of java.lang.NoClassDefFoundError, java.lang.NoSuchMethodError, or something of that sort, and classes with which it happens come from a dependency, it usually means that the project has a dependency conflict.

To resolve the issue, do as follows:

1. Understand which dependency is causing issues.

   It is usually easy to understand from the names of the classes mentioned in the error stack trace. You can search a class name in IDE to find a library to which it belongs or even googled for.

2. Identify where conflicting versions come from.

   Usually, the first thing to do is to run the mvn dependency:tree command in the project root directory and analyze its output. It displays a tree of project dependencies and makes it easy to trace where everything comes from. Sometimes, a conflicting version comes from a different Maven scope. For example, a dependency version used in the test scope takes precedence over one in the compile scope.

3. Decide which version to use to keep everything working.

   This is the hardest thing to do. Sometimes, you should inspect changes between versions and find a non-conflicting one. Sometimes, the version of a dependency that pulls in a conflicting transitive library can be changed to a more recent one. Sometimes, there is no solution at all: no dependency version can be found that is compatible with all places where it is required.

4. Explicitly specify the chosen dependency version in the project pom.xml file.

ODF 2 Bill of Materials

ODF 2 defines a list of dependencies it uses. Versions of these dependencies are not random. They are dictated by the platform version and tested to be compatible with each other and not cause a dependency conflict. This list is called a Bill of Materials (BOM). It is located in the com.workfusion.odf2:odf2-bom module.

All projects created from ODF 2 archetypes use odf2-bom. If a project needs to use a dependency listed in BOM, the version of this dependency can be omitted; in this case, it is taken from BOM.

<dependencyManagement>
    <dependencies>
        <dependency>
            <!-- Here odf2-bom is imported into project -->
            <groupId>com.workfusion.odf2</groupId>
            <artifactId>odf2-bom</artifactId>
            <version>...</version>
            <scope>import</scope>
            <type>pom</type>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <!-- Here dependencies are declared without specifying a version. Versions will be taken from odf2-bom. -->
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-s3</artifactId>
    </dependency>    
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
    </dependency>
</dependencies>