Skip to main content
Version: 10.2.9

Migrate AI DW project based on ODF 2 to JNW

The article delves into the technical specifics of migrating an AI Digital Worker (AI DW) project built with the ODF 2 framework to the solution offered by the Java Native Worker (JNW).

Migrate to ODF 2 v10.2.8​

To use the JNW and run Bot Tasks implemented using the ODF 2 framework, migrate your AI DW project to the ODF 2 v10.2.8 or later. Also, use the latest available versions of bundle-versions-maven-plugin and bundle-maven-plugin:

<odf2.version>10.2.8.20</odf2.version>
<bundle-versions-maven-plugin.version>0.0.28</bundle-versions-maven-plugin.version>
<bundle-maven-plugin.version>10.2.8.8</bundle-maven-plugin.version>

When ODF 2 dependencies are updated to v10.2.8, an AI DW project might stop working properly or compiling. Let's examine what changes are required to fix it.

Binding Reader​

Starting from ODF 2 v10.2.8, BindingReader becomes an interface and does not have some of its methods. The change was required to split the ODF 2 core from using any WebHarvest dependencies and, therefore, let ODF 2 work smoothly with the JNW environment. Namely, BindingReader does not provide access to the WebHarvestTaskItem variable anymore. The Bot Task context data should be obtained from BindingReader directly, for example:

// before 10.2.8
String campaignUuid = bindingReader.getTaskItem().getCampaignDto().getUuid();
String runUuid = bindingReader.getTaskItem().getRun().getUuid();

// after 10.2.8
Optional<UUID> campaignUuid = bindingReader.getBpStepDefinitionId();
Optional<UUID> runUuid = bindingReader.getBpStepId();
note

The JNW does not provide the same set of variables that are available in the Groovy and WebHarvest context. Therefore, the code like bindingReader.getVariable(WebHarvestConstants.WEB_HARVEST_TASK_ITEM, WebHarvestTaskItem.class) always returns an empty variable at the JNW runtime.

ODF 2 WebHarvest integration​

Starting from v10.2.8, ODF 2 introduces a Maven module called odf2-webharvest-integration. This module conveniently packages all code, service implementations, and dependencies crucial for the WebHarvest runtime. If you work with such services, your BCB code most likely stops compiling. You can add the module by introducing the odf2-webharvest-integration artifact to the dependencies section of your BCB. Practically, this brings a project to a pre-10.2.8 state of ODF 2.

<dependency>
    <groupId>com.workfusion.odf2</groupId>
    <artifactId>odf2-webharvest-integration</artifactId>
</dependency>

With the transition towards the JNW approach, there is no need to employ the odf2-webharvest-integration module. In fact, it is imperative that you avoid using it, especially since WebHarvest-based services, such as SecretsVaultService, will not function in the JNW runtime. You also do not need to bring any WebHarvest dependencies to the JNW.

Instead of adding the odf2-webharvest-integration module, adjust your BCB code to ensure it can compile without relying on WebHarvest dependencies. This means the deprecation of any direct WebHarvest components. The ODF 2 framework provides comprehensive abstractions for services, such as Secrets Vault, Billing, S3, and others.

The next step is to stop using ControlTowerServicesModule. This Feather module now comes directly from odf2-webharvest-integration and is specifically tailored for services in the WebHarvest runtime. It is crucial to confirm that none of your Bot Tasks depend on it. For instance:

// before
@Requires({CustomModule.class, DataModule.class, ControlTowerServicesModule.class})
@BotTask
public class MyBotTask implements AdHocTask {}

// after
@Requires({CustomModule.class, DataModule.class})
@BotTask
public class MyBotTask implements AdHocTask {}

OCR Client​

Starting from v10.2.8, ODF 2 removes all OCR dependencies from the ODF 2 core. If you need the integration with OCR, it is recommended to use the OCR Bridge step.

If your code is built on top of the deprecated OCR client and you do not plan to migrate to the OCR Bridge step, add the following dependency manually:

<dependency>
    <groupId>com.workfusion.odf2</groupId>
    <artifactId>odf2-ocr</artifactId>
</dependency>

Billing Info Service​

If you use BillingInfoService obtained from the WebHarvest context, you must switch to the com.workfusion.odf2.service.billing.BillingService abstraction from ODF 2. WebHarvest's BillingInfoService is not a part of the JNW runtime and, therefore, cannot be used.

Before v10.2.8
@BotTask
public class BillingBotTask implements AdHocTask {

    private final BillingInfoService billingInfoService;

    @Inject
    public BillingBotTask(BindingReader bindingReader) {
        this.billingInfoService = bindingReader.getRequiredVariable(WebHarvestConstants.BILLING_SERVICE, BillingInfoService.class);
    }

    @Override
    public TaskRunnerOutput run(TaskInput taskInput) {
        final String jsonData = taskInput.getRequiredVariable("json_billing_data");
        final boolean billingResult = billingInfoService.sendInfo(jsonData);
        return taskInput.asResult().withColumn("billing_result", String.valueOf(billingResult));
    }

}
Starting from v10.2.8
@BotTask
public class BillingBotTask implements AdHocTask {

    private final BillingService billingService;

    @Inject
    public BillingBotTask(BillingService billingService) {
        this.billingService = billingService;
    }

    @Override
    public TaskRunnerOutput run(TaskInput taskInput) {
        final String jsonData = taskInput.getRequiredVariable("json_billing_data");
        final boolean billingResult = billingService.sendInfo(jsonData);
        return taskInput.asResult().withColumn("billing_result", String.valueOf(billingResult));
    }

}
note

For the JNW runtime, you do not have to specify any Feather module to obtain the BillingService implementation. It is provided by JnwIntegrationModule that is added automatically to all Bot Tasks running in the JNW runtime through the ODF 2-JNW integration.

Data Model changes​

Several backward-incompatible changes are introduced to the core data model of the ODF 2 framework, namely to the Transaction object. Therefore, you must change all existing Data Store schemas according to the Transaction entity. Perform the following steps:

  1. Add the skip_until column of the NVARCHAR(255) type to the uc_[code]_transaction_[version] table. If the table is not used in your project, skip it.

  2. Change the split_status column data type to NVARCHAR(255) for the uc_[code]_transaction_[version] table. If the table is not used in your project, skip it.

    <changeSet author="user" id="unique_project_id_1" objectQuotingStrategy="LEGACY">
        <addColumn tableName="uc_[code]_transaction_[version]">
            <column name="skip_until" type="NVARCHAR(255)"/>
        </addColumn>
    </changeSet>

    <changeSet author="user" id="unique_project_id_2" objectQuotingStrategy="LEGACY">
        <modifyDataType columnName="split_status" newDataType="NVARCHAR(255)" tableName="uc_[code]_transaction_[version]"/>
    </changeSet>

  • When working with Liquibase migrations, mind that each table name contains the AI DW code and version. Remember to fill in the code and version values properly.

  • Each changeSet requires an ID attribute, mind the identifier requirements.

JNW module​

For the JNW-related code, create a separate Maven submodule inside your AI DW project. For the majority of AI DWs, the structure of a project looks as follows:

Root pom.xml
| - BCB module
| - JNW module
| - Package module
| - Integration tests module

Complete the following steps:

  1. In any non-project location, create a new project using the JNW archetype and choose the latest JNW version available. The JNW archetype creates a new single-module Maven project. This is crucial to get a proper JNW project structure.

  2. In an IDE, create a new Maven submodule in your AI DW project.

  3. Copy-paste all the code created by the JNW archetype into the submodule.

  4. Modify the submodule's pom.xml so that it looks like as in the listing below:

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        
        <!-- Refers to the root pom.xml with 'com.workfusion.dw:my-dw-project:1.0.0' GAV coordinates -->
        <parent>
            <groupId>com.workfusion.dw</groupId>
            <artifactId>my-dw-project</artifactId>
            <version>1.0.0</version>
        </parent>

        <!-- JNW submodule artifact ID -->
        <artifactId>my-dw-project-jnw</artifactId>

        <!-- Prefix name for all bot configs created from this module -->
        <name>My AI DW Project JNW</name>
        
        <!-- uncomment for ODF 10.2.8
        <properties>
            <!-- Use latest versions available -->
            <wf.jnw-toolkit.version>1.0.0.15</wf.java-native-worker-parent-bom.version>
            <wf.runtime-logger-configurer.version>1.0.142</wf.runtime-logger-configurer.version>
        </properties>

        <dependencyManagement>
            <dependencies>
               <dependency>
                    <groupId>com.workfusion.jnw.toolkit</groupId>
                    <artifactId>jnw-toolkit-bom</artifactId>
                    <version>${wf.jnw-toolkit.version}</version>
                    <type>pom</type>
                    <scope>import</scope>
                </dependency>
            </dependencies>
        </dependencyManagement>
        -->

        <!-- uncomment for ODF 10.2.9
        <properties>
            <!-- Use latest versions available -->
            <wf.java-native-worker-parent-bom.version>1.0.1.12</wf.java-native-worker-parent-bom.version>
            <wf.java-native-worker-base.version>1.0.1.13</wf.java-native-worker-base.version>
            <wf.runtime-logger-configurer.version>1.0.136</wf.runtime-logger-configurer.version>
        </properties>
     
        <dependencyManagement>
            <dependencies>
                <dependency>
                    <groupId>com.workfusion.spa.java.native.worker</groupId>
                    <artifactId>java-native-worker-parent</artifactId>
                    <version>${wf.java-native-worker-parent-bom.version}</version>
                    <type>pom</type>
                    <scope>import</scope>
                </dependency>
                <dependency>
                    <groupId>com.workfusion.spa.java.native.worker</groupId>
                    <artifactId>worker-base</artifactId>
                    <version>${wf.java-native-worker-base.version}</version>
                    <type>pom</type>
                    <scope>import</scope>
                </dependency>
            </dependencies>
        </dependencyManagement>
        -->

        <dependencies>
            <!-- Refers to project's BCB module -->
            <dependency>
                <groupId>com.workfusion.dw</groupId>
                <artifactId>my-dw-project-bcb</artifactId>
                <version>${project.version}</version>
            </dependency>

            <!-- ODF2-JNW integration dependencies -->
            <dependency>
                <groupId>com.workfusion.odf2</groupId>
                <artifactId>odf2-jnw-integration</artifactId>
            </dependency>
            <dependency>
                <groupId>com.workfusion.odf2</groupId>
                <artifactId>odf2-jnw-compiler</artifactId>
                <scope>provided</scope>
            </dependency>

            <!-- JNW dependencies -->
            <dependency>
                <groupId>com.workfusion.spa.java.native.worker</groupId>
                <artifactId>java-native-worker-core</artifactId>
            </dependency>
            <dependency>
                <groupId>com.workfusion.util</groupId>
                <artifactId>runtime-logger-configurer</artifactId>
                <version>${wf.runtime-logger-configurer.version}</version>
            </dependency>

            <!-- JDBC driver for Data Store operations -->
            <dependency>
                <groupId>com.microsoft.sqlserver</groupId>
                <artifactId>mssql-jdbc</artifactId>
                <version>7.0.0.jre8</version>
            </dependency>

            <!-- Test dependencies -->
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <scope>test</scope>
                <exclusions>
                    <exclusion>
                        <groupId>org.apache.logging.log4j</groupId>
                        <artifactId>*</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
            <dependency>
                <groupId>com.h2database</groupId>
                <artifactId>h2</artifactId>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>com.workfusion.odf2</groupId>
                <artifactId>odf2-junit</artifactId>
                <scope>test</scope>
            </dependency>
        </dependencies>

        <build>
            <plugins>
                <!-- Use the same 'spring-boot' version as in 'java-native-worker-base' project -->
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                    <version>2.7.12</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>repackage</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
                
                <!-- Settings required by the 'odf2-jnw-compiler' -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <configuration>
                        <compilerArgs>
                            <arg>-AgroupId=${project.groupId}</arg>
                            <arg>-AartifactId=${project.artifactId}</arg>
                            <arg>-Aversion=${project.version}</arg>
                        </compilerArgs>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </project>

  5. Build the project. Make sure there are no issues and the project is built successfully.

Task Processors​

In the JNW environment, the equivalent of an ODF 2 Bot Task is referred to as a Task Processor.

For each Bot Task you intend to employ in the JNW runtime, it is essential to have a corresponding JNW Task Processor. For each Task Processor, provide an XML Bot Config.

The odf2-jnw-integration module furnishes all requisite dependencies, configurations, and tools to facilitate a seamless transition to JNW Task Processors. In addition to that, the odf2-jnw-compiler module offers the auto-generation of a JNW Task Processor and an XML Bot Config.

Auto-generation of JNW compatibility classes and files​

ODF 2 offers the auto-generation of the JNW compatibility layer.

Let's migrate a simple Bot Task that looks as follows:

@BotTask
public class MyBotTask implements AdHocTask {

    @Override
    public TaskRunnerOutput run(TaskInput taskInput) {
        return taskInput.asResult();
    }

}

To take advantage of the code generation mechanism provided by odf2-jnw-compiler, annotate the MyBotTask class with the following annotation:

@BotTask
@AutoJnwCompatibilityComponent(autoTaskProcessor = @AutoTaskProcessor(id = "my-bot-task", className = "MyBotTaskProcessor"))
public class MyBotTask implements AdHocTask {

    @Override
    public TaskRunnerOutput run(TaskInput taskInput) {
        return taskInput.asResult();
    }

}

For more details, see Generate code with JNW Toolkit. A Task Processor is generated with the MyBotTaskProcessor class and the ID of my-bot-task as specified in the annotation. If the parameters are omitted, the annotation processor generates names based on the name of the Bot Task class. Additionally, an XML Bot Config is generated.

You can find generated classes in the target/generated-sources/annotations directory and XML files in the target/classes/configs/main folder of your JNW submodule.

Manual creation of JNW Task Processor​

You can also create a JNW Task Processor manually.

For example, you need to migrate a simple Bot Task that looks as follows:

@BotTask
public class MyBotTask implements AdHocTask {

    @Override
    public TaskRunnerOutput run(TaskInput taskInput) {
        return taskInput.asResult();
    }

}

To create a Task Processor for MyBotTask, do as follows:

  1. Create a new Java class in the JNW submodule that implements the ITaskProcessor interface.

  2. Annotate the class with @TaskProcessor.

  3. Inject the OdfJnwIntegration bean (already available in the Spring context) using a class constructor.

  4. Call the executeOdfTask method and pass the MyBotTask class with the TaskInputData object.

    @TaskProcessor(id = "my-bot-task")
    public class MyBotTaskProcessor implements ITaskProcessor {

        private final OdfJnwIntegration odfJnwIntegration;

        public MyBotTaskProcessor(OdfJnwIntegration odfJnwIntegration) {
            this.odfJnwIntegration = odfJnwIntegration;
        }

        @Override
        public TaskOutputData process(TaskInputData inputData) {
            return odfJnwIntegration.executeOdfTask(MyBotTask.class, inputData);
        }

    }

Thus, create a separate Task Processor for each Bot Task (the class annotated with @BotTask) you plan to use in your Business Process (BP).

note

If a Bot Task is annotated with @SendToExternalConnector, put the @SendToExternalConnector annotation to a respective Task Processor.

For each Task Processor, provide an XML Bot Config that typically looks as follows:

<?xml version="1.0" encoding="UTF-8"?>
<config type="java">
    <worker>${GAV}</worker>
    <processor>${processorId}</processor>
</config>

The ODF 2 framework comes with the odf2-jnw-compiler module. It creates XML Bot Configs automatically for all classes annotated with @TaskProcessor during the compile phase of your project. You can find generated XML files in the target/classes/configs/main folder of your JNW submodule.

By default, a name for an XML Bot Config file is derived from a class name (MyBotTaskProcessor.class > my-bot-task-processor.xml). If you want a custom name for a generated XML file, use the @FileName annotation:

@FileName("my-custom-name.xml")
@TaskProcessor(id = "my-bot-task")
public class MyBotTaskProcessor implements ITaskProcessor {}

Testing Task Processors​

As the JNW fundamentally operates as a Spring Boot application, you can take advantage of the utilities, such as spring-boot-test and other test libraries. The JNW archetype offers an initial configuration for testing Task Processors.

Here is an example of a Task Processor test from the Tara project:

class RequestFilterHitsTaskProcessorTest extends BaseProcessorTest {

    @Autowired TaskProcessingRouter router;
    @Autowired PssOrmSupport ormSupport;

    private TransactionRepository transactionRepository;
    private OrmLiteRepository<PssDataModelRecords> recordsRepository;

    @BeforeEach
    void setUp() {
        ormSupport.createPssTables(Transaction.class, PssDataModelRecords.class);

        transactionRepository = ormSupport.getTransactionRepository();
        recordsRepository = ormSupport.getRepository(PssDataModelRecords.class);
    }

    @AfterEach
    void tearDown() {
        ormSupport.dropPssTables();
    }

    @Test
    @DisplayName("should filter hits")
    void shouldFilterHits() throws IOException {
        // given
        Transaction transaction = transactionRepository.startNewTransaction("NEW");
        String requestJson = getResourceAsString("/it-data/request-filter-hits-message.json");

        createPssRecord(transaction.getUuid().toString(), requestJson);

        TaskInput<TaskInputWithContext> taskInput = new TaskDataPreparation(RequestFilterHitsTaskProcessor.class)
                .withTransaction(transaction)
                .withInputData(ProcessField.MESSAGE_DATASTORE.toString(), recordsRepository.getDao().getTableName())
                .build();

        // when
        TaskResult<TaskOutputData> result = router.execute(taskInput);

        // then
        TaskResultAssert.assertThat(result).isSuccessful()
                .extractingRowsByKey(ProcessField.MESSAGE_TYPE.toString())
                .containsExactly(MessageType.PAYMENT_TRANSFER.toString());

        String messageAfterUpdate = recordsRepository.findAll().get(0).getMessageJson();

        Message expectedMessage = new PssMessageConverter().readMessageFromJson(requestJson);
        Message actualMessage = new PssMessageConverter().readMessageFromJson(messageAfterUpdate);

        assertThat(actualMessage).isEqualTo(expectedMessage);
    }

    private PssDataModelRecords createPssRecord(String requestId, String requestJson) {
        return recordsRepository.create(PssDataModelRecords.builder()
                .requestId(requestId)
                .requestJson(requestJson)
                .build());
    }

    private String getResourceAsString(String resourceName) throws IOException {
        URL url = Objects.requireNonNull(getClass().getResource(resourceName), String.format("Resource not found: %s", resourceName));
        return IOUtils.toString(url, UTF_8);
    }

}

BCB changes​

If your BCB is configured to function with the PF4J classloading isolation, you can eliminate the corresponding configurations. This is because, inherently, JNW doesn't require any form of classloading isolation. Although it is not an absolute requirement, taking this step reduces the build time and results in a smaller size of the final artifact. The size reduction can be quite noticeable depending on the dependencies your BCB relies on.

To get rid of PF4J-related configurations, locate the declaration for the maven-assembly-plugin in the BCB's pom.xml file and completely remove it, along with the src/main/resources/assembly.xml file. Typically, the section you need to delete appears as follows:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-assembly-plugin</artifactId>
    <configuration>
        <descriptors>
            <descriptor>src/main/resources/assembly.xml</descriptor>
        </descriptors>
        <archiverConfig>
            <compress>false</compress>
        </archiverConfig>
        <archive>
            <index>true</index>
            <manifest>
                <addClasspath>true</addClasspath>
                <classpathPrefix>lib/</classpathPrefix>
            </manifest>
            <manifestEntries>
                <Plugin-Id>${project.groupId}:${project.artifactId}:${project.version}</Plugin-Id>
                <Plugin-Cache-Supported>true</Plugin-Cache-Supported>
            </manifestEntries>
        </archive>
        <appendAssemblyId>false</appendAssemblyId>
    </configuration>
    <executions>
        <execution>
            <id>create-archive</id>
            <phase>package</phase>
            <goals>
                <goal>single</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Package structure​

The outcome of the JNW submodule is a Spring Boot application–a single JAR file that incorporates all necessary dependencies, including the project's BCB. As a result, there's no longer a need to package the BCB itself in an AI DW bundle. Instead, a JNW JAR file must be packaged.

See an example of a package.xml file that packages JNW's JAR file into the artifactory-dependency/workers folder:

<?xml version="1.0" encoding="UTF-8"?>
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.0.0 http://maven.apache.org/xsd/assembly-2.0.0.xsd">
    <id>assembly-id</id>

    <includeBaseDirectory>false</includeBaseDirectory>

    <formats>
        <format>zip</format>
    </formats>

    <fileSets>
        <fileSet>
            <directory>src/main/resources</directory>
            <outputDirectory/>
        </fileSet>
    </fileSets>

    <dependencySets>
        <dependencySet>
            <outputDirectory>artifactory-dependency/workers/com/workfusion/dw/my-dw-project-jnw/${project.version}</outputDirectory>
            <includes>
                <include>com.workfusion.dw:my-dw-project-jnw:jar:${project.version}</include>
            </includes>
            <useTransitiveDependencies>false</useTransitiveDependencies>
            <useStrictFiltering>true</useStrictFiltering>
            <useProjectArtifact>false</useProjectArtifact>
        </dependencySet>
    </dependencySets>
</assembly>

Business Process changes​

The final step involves transitioning Bot Tasks to Task Processors in your BP. Here's how you can do it:

  1. Import the migrated AI DW project to a Control Tower instance.
  2. Open the BP you intend to convert.
  3. Modify each bot configuration in the BP, replacing the ODF 2 Bot Task with the corresponding JNW Task Processor.
  4. Save and export the updated BP.
  5. Store the exported BP in the AI DW's package module.
note

Once a project is migrated to the JNW, it is no longer possible to continue using ODF 2 Bot Tasks in their original form. Even if you attempt to integrate ODF 2 Bot Tasks into a BP, it is unlikely to function properly due to the alterations made during the JNW migration process.