Skip to main content
Version: 10.3.1

Java Native Worker

The Java native mechanism enables you to write and deploy custom pure Java-based Workers with any libraries without additional loading or linking.

Creating a Java Native Worker (JNW) consists of the following stages:

  1. Generate a Worker application from the archetype.
  2. Add necessary modules. If you are upgrading an existing worker to Work.AI v10.3.0+ and your application uses module-secrets v2.0.0.5 or earlier, see Migrate module-secrets configuration.
  3. Implement task functionality in the form of Java classes. Add needed third-party dependencies if required.
  4. Build a Worker locally.
  5. Test tasks using the worker-task-test library.
  6. Create a Business Process (BP) with Java Worker tasks.
  7. Test the BP. You can do it in one of the following ways:
  8. Modify, update, re-deploy, or test Workers as needed.

JNW compatibility matrix​

See the version compatibility matrix in the table below.

Work.AIJNW toolkitworker-task-testjava-native-worker-parent-bomjava-native-worker-archetypejava-native-worker-bundle-archetype
10.33.0.0.492.0.37.02.1.0.28.02.1.0.351.0.1.108.0

Migrate module-secrets configuration​

info

The steps are required only when upgrading the Work.AI platform to version 10.3.0 or later for existing Java Native Workers or Triggers that use module-secrets v2.0.0.5 or earlier.

Before migrating the module-secrets module, ensure that the following conditions are met:

  • Your application uses the Java Native Worker or Trigger with module-secrets v2.0.0.5 or earlier.
  • Your target platform environment version is v10.3.0 or later.
  • Your application uses the module-secrets module.

To complete the upgrade of an existing Java Native Worker or Trigger to Work.AI v10.3.0+, follow the steps below:

  1. Update the wdt-bom dependency in your project to v3.0.0.10 or the latest available version.

  2. Create the wf-env-configuration.json file at the following location: src/main/resources/META-INF/worker/wf-env-configuration.json.

  3. Populate wf-env-configuration.json with the required content for your new environment: 

    {
      "name": "jnw",
      "env": {
        "WF_APP_VAULT_ROLE_R_ID": "wf.vault.wf.app.user.role.id",
        "WF_APP_VAULT_ROLE_S_ID": "wf.vault.wf.app.user.role.secret.id",
        "WF_CLIENT_VAULT_R_ID": "wf.vault.wf.app.user.role.id",
        "WF_CLIENT_VAULT_S_ID": "wf.vault.wf.app.user.role.secret.id"
      }
    }

  4. Modify the application.yml file located at src/main/resources/application.yml. Add the following properties to enable and configure Vault integration:

    spring:
      cloud:
        vault:
          uri: ${WF_APP_VAULT_URI}
          kv:
            enabled: true
            backend: config                   #BASE_MOUNT_PATH (default: config)
            profile-separator: '/'
            default-context: application      #DEFAULT-CONTEXT (default: application)
          authentication: approle             #Authentication method (default: approle)
          app-role:
            role-id: ${WF_APP_VAULT_ROLE_R_ID}
            secret-id: ${WF_APP_VAULT_ROLE_S_ID}
      config:
        import:
          - "optional:zookeeper:"
          - "optional:vault://"  # Enables Vault property source out of the box.
          - "optional:classpath:wf-module-secrets-application.yml"  # Enables module-secrets property source.

  5. Rebuild your project to ensure all changes are applied.

Add modules​

Only the module-event module that provides the "send the event to CT" component is default added as a dependency in the generated application.

In frequent cases, you might need the following modules:

  • module-secrets to access secrets from Vault.
note

By default, you can access configuration parameters in Vault using standard spring mechanisms (@Value annotation, and so on), and you don't need to access this module. It is required only to access custom secrets defined in the CT interface.

  • module-billing to send billing information from a step.

See details on each module below:

Event Module is a project that defines all necessary classes and APIs used to log events from the JNW into BEP.

To apply the module, do as follows:

  1. Add a dependency to your project's pom.xml. Note that for projects created using the JNW archetype, this module is already included in pom.xml.

    <dependencies>
        <dependency>
            <groupId>com.workfusion.spa.java.native.worker</groupId>
            <artifactId>module-event</artifactId>
        </dependency>

  2. Inject the event logger into the component:

    import org.slf4j.Logger;
    import com.workfusion.spa.jnative.worker.module.event.EventLogger;

    @TaskProcessor(id = "sample-task")
    public class SampleTaskProcessor implements ITaskProcessor {

        private final Logger logger;

        @Autowired
        public SampleTaskProcessor(EventLogger workerLogger) {
            this.logger = workerLogger.getEventLogger();
        }

        @Override
        public TaskOutputData process(TaskInputData input) throws TaskProcessorException {
            try {
                // ...
                //send log by worker logger into Control Tower as an event.
                logger.info("Some event info for CT side.");
                // ...
            } catch (Exception e) {
                //send error event to CT and re-throw exception
                logger.error(e.getMessage(), e);
                throw e;
            }
        }
    }

The modules are added to pom.xml but commented. To use them, uncomment the corresponding lines:

<dependencies>
    <dependency>
        <groupId>com.workfusion.spa.java.native.worker</groupId>
        <artifactId>module-event</artifactId>
    </dependency>

    <!--uncomment next dependency, if module for sending billing info is needed-->
    <!--<dependency>
        <groupId>com.workfusion.spa.java.native.worker</groupId>
        <artifactId>module-billing</artifactId>
    </dependency>-->

    <!--uncomment next dependency, if a module, which works with client secrets, is needed-->
    <!--<dependency>
        <groupId>com.workfusion.spa.java.native.worker</groupId>
        <artifactId>module-secrets</artifactId>
    </dependency>-->
</dependencies>

You can add other modules manually in the same way as you add any other Maven dependencies:

<dependencies>
    <!-- ... -->

    <dependency>
        <groupId>com.workfusion.spa.java.native.worker</groupId>
        <artifactId>module-with-some-functionality</artifactId>
        <version>1.2.3</version>
    </dependency>
</dependencies>

Check Worker artifact size​

During the development process, you might need to add third-party Maven dependencies to implement specific functionality in your JNW. It is crucial to carefully manage this process by adding only essential dependencies and excluding any unnecessary or unused transitive dependencies from those you include.

Keeping the resulting Worker artifact (JAR) as small as possible is vital. A smaller Worker artifact ensures shorter startup time and reduces network, NFS, and CPU load on Master and Agent hosts.

warning

It is strongly recommended to keep the resulting Worker artifact JAR size under 300 MB. Failure to adhere to this recommendation can significantly degrade the Platform's performance and stability under load.

Implement task functionality​

To implement the Worker functionality, add the ITaskProcessor classes: one class per each task type.

In the generated application, the archetype creates SampleTaskProcessor in the task sub-package. You can use SampleTaskProcessor as an example or a starter for task processors. This sample task already contains an injected logger for sending events to CT.

@TaskProcessor(id = "sample-task")
public class SampleTaskProcessor implements ITaskProcessor {

    //Java Native Worker logger, all messages will be sent into CT as an event.
    private final Logger logger;
    
    @Autowired
    public SampleTaskProcessor(EventLogger workerLogger) {
        this.logger = workerLogger.getEventLogger();
    }

    @Override
    public TaskOutputData process(TaskInputData input) throws TaskProcessorException {
        try {

            //task processing logic

            //send log by worker logger into Control Tower as an event.
            logger.info("Some event info for CT side.");


            //build output
            return new TaskOutputData()
                    .addRow(new TaskOutputRow()
                            //.put("some_field", "some_value")
                            //.put("other_field", "other_value")
                    )
                    //add more result rows if required
            ;
        } catch (Exception e) {
            logger.error(e.getMessage(), e);
            throw e;
        }
    }

}

A Task processor is a spring component, so you can use any spring mechanism in it, for example:

  • Auto-wiring
  • Post-construct and per-destroy
  • Access environment and configuration, and so on

Build Worker locally​

You can build a Worker with the standard maven mvn clean install command. After building it, you get a Spring Boot application artifact in the /target directory. You can deploy the artifact to a target environment as any other Worker's artifacts.

Create BP with Java Worker tasks​

To add tasks to the JNW, perform the following steps:

  1. Create a BP.
  2. Add a Bot Task and edit its source:
    1. Add type="java" to the root <config> tag.
    2. Remove everything inside <config> and populate the tag with the Java native task metadata.
  3. Save the step.
  4. Save the BP.

The JNW tasks have the minimum structure:

<?xml version="1.0" encoding="UTF-8"?>
<config type="java">
    <worker>${GAV}</worker>
    <processor>${processorId}</processor>
</config>

Here, replace:

  • ${GAV} with your Worker's current groupId:artifactId:version. You can find this parameter in pom.xml.
  • ${processordId} with the corresponding task processor ID. See the id attribute in the @TaskProcessor annotation in a corresponding class.

For example, groupId:artifactId:version ($GAV) in your Worker's pom.xml looks as follows:

<project>
    <groupId>my.comp</groupId>
    <artifactId>my-worker</artifactId>
    <version>1.2.3</version>
    ...

You create two task processors:

@TaskProcessor(id = "first-task-type")
public class FirstTaskProcessor implements ITaskProcessor { ... }

@TaskProcessor(id = "second-task-type")
public class SecondTaskProcessor implements ITaskProcessor { ... }

In this case, the tasks are as follows:

  • For FirstTaskProcessor:

    <?xml version="1.0" encoding="UTF-8"?>
    <config type="java">
        <worker>my.comp:my-worker:1.2.3</worker>
        <processor>first-task-type</processor>
    </config>

  • For SecondTaskProcessor:

    <?xml version="1.0" encoding="UTF-8"?>
    <config type="java">
        <worker>my.comp:my-worker:1.2.3</worker>
        <processor>second-task-type</processor>
    </config>

You can use the following Task parameters:

ParameterRequiredDescription
workerYesgroupId:artifactId:version of the JNW artifact.
processorYesTask processor ID. The code is used to find a processor class and route a task to it.
splitDataNoParameter that defines the split-data behavior. Values:
  • auto (default): automatically detects split behavior: If the result contains a single record, do not split. If the result contains two or more records, split data
  • force: always split data. Applicable for the monitoring-loop flow, and so on.
sendResultToCallerNoParameter that defines if CT must send results to an external connector (for example, REST connector), a caller BP, and so on. Implements the same logic as the send-to-external-connector attribute in export-plugin. Values:
  • false (default): do not try to send results from this step.
  • true: try to send results from this step to an external connector, a caller BP, and so on.

Run BP to test logic​

Deploy Worker to CT​

To test the JNW, deploy it to a target environment and run the corresponding BP. You can deploy the Worker by importing the artifact as a part of the AI Agent's package.

Run Worker from IDE in debug mode​

You can run the Worker manually to debug it in IDE. In this case, provide the minimum set of configuration parameters.

Full environment​

To run the Worker against a full environment, do the following preparations and preconditions checks:

  1. Make sure the Worker is not deployed to Nexus on the environment to prevent WMS from starting on BEP cluster Worker instances that "steal" tasks from your locally running Worker. If the Worker is already deployed on Nexus, you must perform one of the following actions:

    • Delete the Worker artifact from Nexus and the shared directory.

    • Stop WMS and kill custom Workers (if any) in Marathon. Note that this prevents starting any Worker on a BEP cluster. Thus, if your BP has mixed step types, it will not work.

  2. Download the client certificate /opt/workfusion/ssl/vault_workfusion.p12 for Vault to a local machine.

  3. Copy the secure.storage.* properties from the /opt/workfusion/conf/workfusion.properties file. Add the properties to the Worker configuration parameters. Replace the value for secure.storage.client.certificate with a local path to the downloaded vault_workfusion.p12.

  4. Run the Worker with the following parameters:

    spring.cloud.zookeeper.connect-string=${master-host}:2181
    execution.task.queue.input=exec.${ct-client-id}.worker.${gav}.DEFAULT

    secure.storage.type=VAULT
    # ... and other 'secure.storage.*' parameters

  5. Run the BP.

Deploy Worker to Work.AI platform​

The JNW, like any other Worker types (CT, AutoML, OCR), must be deployed to Nexus into the workers repository, so WMS can download and start it.

To deploy Java Native Workers to the Work.AI platform, import the Worker artifact as a part of an AI Agent Asset Bundle.

You can import the JNW artifact together with a Bot Config, where it is used, with the help of the Asset Bundle import mechanism. In this case, two components are used:

Import artifact​

Before importing the artifact, ensure that you have a JAR file with some groupId:artifactId:version names.

The following instruction is applicable for building a minimum standalone bundle from scratch. The bundle contains only a JNW artifact or for adding Worker artifact to some existing bundle with repackage (steps 3, 4, 5, 7).

To import the Worker artifact, perform the following steps:

  1. Create the artifactory-dependency directory in the bundle if it does not exist.

  2. In the artifactory-dependency directory, create the workers directory as the first underlying directory name. It represents the name of the Nexus repository where the artifact must be imported.

  3. In the workers directory, create the tree of $GAV directories. The group name must be split into the hierarchy of sub-directories by a dot in the group name, for example, com.supertest.

  4. In the version directory, place the JAR artifact named according to the convention: artifact_name-version.jar.

  5. In the bundle's root directory (the same layer as the artifactory-dependency directory), create the meta-info.json file. See the structure and description in Package assets into AI Agent Asset Bundle.

    artifactory-dependency
    │   └── workers
    │   │       └── com
    │   │           ├── supertest
    │   │               └── test-native-worker
    │   │                   └── 1.0
    │   │                       └── test-native-worker-1.0.jar

    In this case, Worker's GAV is com.supertest:test-native-worker:1.0.

  6. Archive the above structure into a zip archive. Your bundle is now ready.

note

The workers Nexus repository is a release repository. You can't import an artifact containing the SNAPSHOT postfix in its version. The release artifact versions are required.

Import Bot Config​

When importing a Bot Config, you have the following options:

Import from Java Native Worker artifact​

A Bot Config that contains the JNW processor call can be imported directly from the JNW artifact.

Place the Bot config XMLs in the src/main/resources/configs/main directory as shown below:

my-custom-worker
     └──src
          └──main
               └──resources
                      └──configs
                           └──main
                                ├──email_intake.xml
                                └──validation.xml

When you build the JNW and import it as a part of the Asset Bundle, machine configs are also imported to CT.

You can only import machine configs that call to the JNW processors in such a way. This approach makes importing WebHarvest or some arbitrary Bot Config Bundle impossible.

Import from Asset Bundle​

A Bot Config (step) configured to use the JNW processing can be imported similarly to any other.

Before importing a Bot Config from the Asset Bundle, ensure you have the bot-config.xml file exported previously from another instance.

To import a Bot Config, do as follows:

  1. Create the bot-config directory if it doesn't exist in a bundle.

  2. Place the XML bot config file in the bot-config directory.

  3. Create the meta-info.json file as described in the previous section.

  4. Archive the above files and directories into the ZIP archive. Your bundle is now ready.

    bot-config
    │   └── some-config.xml

See the example in some-config.xml.

info

The config contains UUIDs, names, and other components taken from the original instance. If you manually modify its machine configuration in a text editor or export the same step with different machine code configurations without changing UUIDs, the bot config machine code will be overwritten during the import if an existing configuration is present with identical IDs.

The bundle structure with the JNW and a bot configuration looks as follows:

bot-config
│   └── some-config.xml
artifactory-dependency
│   └── workers
│   │       └── com
│   │           ├── supertest
│   │               └── test-native-worker
│   │                   └── 1.0
│   │                       └── test-native-worker-1.0.jar

Select Java version​

WMS supports running BEP Workers on different versions of JDK. Currently, JDK versions 8 and 17 are supported. If the version is not defined, JDK 8 is used.

To set a target JDK version, add the following lines to the META-INF/worker/worker.yml file:

command: ...
app-name: ...
// ... other configuration parameters
java-version: 17
info

java-version must be a high-level parameter without any indent in YAML.