Java Native Worker
The Java native mechanism enables you to write and deploy custom pure Java-based Workers with any libraries without additional loading or linking.
Creating a Java Native Worker (JNW) consists of the following stages:
- Generate a Worker application from the archetype.
- Add necessary modules. If you are upgrading an existing worker to Work.AI v10.3.0+ and your application uses
module-secretsv2.0.0.5 or earlier, see Migratemodule-secretsconfiguration. - Implement task functionality in the form of Java classes. Add needed third-party dependencies if required.
- Build a Worker locally.
- Test tasks using the
worker-task-testlibrary. - Create a Business Process (BP) with Java Worker tasks.
- Test the BP. You can do it in one of the following ways:
- Deploy the Worker to Control Tower (CT).
- Run the Worker from the IDE in the debug mode.
- Modify, update, re-deploy, or test Workers as needed.
JNW compatibility matrix
See the version compatibility matrix in the table below.
| Work.AI | WDT | worker-task-test | java-native-worker-parent-bom | java-native-worker-archetype | java-native-worker-bundle-archetype |
|---|---|---|---|---|---|
| 10.3.2 | 3.0.0.78.3 | 2.0.57.1 | 2.1.0.46.1 | 2.1.0.63.3 | 1.0.1.137.3 |
Trigger compatibility matrix
See the version compatibility matrix in the table below.
| Work.AI | WDT | worker-task-test | trigger-core | trigger-worker-archetype | trigger-bundle-archetype |
|---|---|---|---|---|---|
| 10.3.1 | 3.0.0.78.3 | 2.0.57.1 | 2.0.44.0 | 2.0.0.53.1 | 1.0.0.26.3 |
Migrate module-secrets configuration
The steps are required only when upgrading the Work.AI platform to version 10.3.0 or later for existing Java Native Workers or Triggers that use module-secrets v2.0.0.5 or earlier.
Before migrating the module-secrets module, ensure that the following conditions are met:
- Your application uses the Java Native Worker or Trigger with
module-secretsv2.0.0.5 or earlier. - Your target platform environment version is v10.3.0 or later.
- Your application uses the
module-secretsmodule.
To complete the upgrade of an existing Java Native Worker or Trigger to Work.AI v10.3.0+, follow the steps below:
Update the
wdt-bomdependency in your project to v3.0.0.10 or the latest available version.Create the
wf-env-configuration.jsonfile at the following location:src/main/resources/META-INF/worker/wf-env-configuration.json.Populate
wf-env-configuration.jsonwith the required content for your new environment:{
"name": "jnw",
"env": {
"WF_APP_VAULT_ROLE_R_ID": "wf.vault.wf.app.user.role.id",
"WF_APP_VAULT_ROLE_S_ID": "wf.vault.wf.app.user.role.secret.id",
"WF_CLIENT_VAULT_R_ID": "wf.vault.wf.app.user.role.id",
"WF_CLIENT_VAULT_S_ID": "wf.vault.wf.app.user.role.secret.id"
}
}Modify the
application.ymlfile located atsrc/main/resources/application.yml. Add the following properties to enable and configure Vault integration:spring:
cloud:
vault:
uri: ${WF_APP_VAULT_URI}
kv:
enabled: true
backend: config #BASE_MOUNT_PATH (default: config)
profile-separator: '/'
default-context: application #DEFAULT-CONTEXT (default: application)
authentication: approle #Authentication method (default: approle)
app-role:
role-id: ${WF_APP_VAULT_ROLE_R_ID}
secret-id: ${WF_APP_VAULT_ROLE_S_ID}
config:
import:
- "optional:zookeeper:"
- "optional:vault://" # Enables Vault property source out of the box.
- "optional:classpath:wf-module-secrets-application.yml" # Enables module-secrets property source.Rebuild your project to ensure all changes are applied.
Add modules
Only the module-event module that provides the "send the event to CT" component is default added as a dependency in the generated application.
In frequent cases, you might need the following modules:
module-secretsto access secrets from Vault.
By default, you can access configuration parameters in Vault using standard spring mechanisms (@Value annotation, and so on), and you don't need to access this module. It is required only to access custom secrets defined in the CT interface.
module-billingto send billing information from a step.
See details on each module below:
- module-event
- module-secrets
- module-billing
- module-workspace
- module-distributed-services
Event Module is a project that defines all necessary classes and APIs used to log events from the JNW into BEP.
To apply the module, do as follows:
Add a dependency to your project's
pom.xml. Note that for projects created using the JNW archetype, this module is already included inpom.xml.<dependencies>
<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-event</artifactId>
</dependency>Inject the event logger into the component:
import org.slf4j.Logger;
import com.workfusion.spa.jnative.worker.module.event.EventLogger;
@TaskProcessor(id = "sample-task")
public class SampleTaskProcessor implements ITaskProcessor {
private final Logger logger;
@Autowired
public SampleTaskProcessor(EventLogger workerLogger) {
this.logger = workerLogger.getEventLogger();
}
@Override
public TaskOutputData process(TaskInputData input) throws TaskProcessorException {
try {
// ...
//send log by worker logger into Control Tower as an event.
logger.info("Some event info for CT side.");
// ...
} catch (Exception e) {
//send error event to CT and re-throw exception
logger.error(e.getMessage(), e);
throw e;
}
}
}
Secrets Module is a project that defines all needed classes and APIs used for getting secrets from HashiCorp Vault.
To apply the module, do as follows:
In your JNW, add or uncomment the required dependencies. In the example below, you can uncomment the
<module-secrets>dependency if a module that works with client secrets is needed.<dependencies>
<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-event</artifactId>
</dependency>
<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-secrets</artifactId>
</dependency>
</dependencies>Check that the
secure.storage.safe.customer.defaultproperty is defined in the ZooKeeper context. This value configured during the installation procedure must be the same as in CT.
See the example of using the secret service:
import java.util.Optional;
import org.slf4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import com.workfusion.spa.jnative.worker.core.api.TaskInputData;
import com.workfusion.spa.jnative.worker.core.api.TaskOutputData;
import com.workfusion.spa.jnative.worker.core.api.TaskOutputRow;
import com.workfusion.spa.jnative.worker.core.task.ITaskProcessor;
import com.workfusion.spa.jnative.worker.core.task.TaskProcessor;
import com.workfusion.spa.jnative.worker.core.task.TaskProcessorException;
import com.workfusion.spa.jnative.worker.module.event.EventLogger;
import com.workfusion.spa.jnative.worker.module.secret.SecretService;
import com.workfusion.spa.jnative.worker.module.secret.entity.SecretDto;
@TaskProcessor(id = "secret-task")
public class SecretTaskProcessor implements ITaskProcessor {
//Java Native Worker logger, all messages will be sent into CT as an event.
private final Logger logger;
//Java Native Worker client secret service.
private final SecretService secretService;
@Autowired
public SecretTaskProcessor(EventLogger workerLogger, SecretService secretService) {
this.logger = workerLogger.getEventLogger();
this.secretService = secretService;
}
@Override
public TaskOutputData process(TaskInputData input) throws TaskProcessorException {
try {
// create client secret with provided data.
secretService.createSecret("my-custom-identifier", "secret-name", "secret-value");
// update client secret, which will be created above.
secretService.updateSecret("my-custom-identifier", "updated-name", "updated-value");
// reset secret value for provided alias.
secretService.resetSecret("my-custom-identifier");
// get client secret with provided alias.
Optional<SecretDto> optionalSecret = secretService.getSecret("my-custom-identifier");
// delete client secret for provided alias
secretService.deleteSecret("my-custom-identifier");
//task processing logic
//build output
return new TaskOutputData()
.addRow(new TaskOutputRow()
//.put("some_field", "some_value")
//.put("other_field", "other_value")
)
//add more result row if required
;
} catch (Exception e) {
logger.error(e.getMessage(), e);
throw e;
}
}
}
Billing Module is a project that defines all necessary classes and APIs used to log billing information from the JNW into BEP.
To apply the module, do as follows:
Add a dependency to your project's
pom.xml. Note that for projects created using the JNW archetype, this module is already included inpom.xml:<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-billing</artifactId>
</dependency>Inject the billing service into the component:
import org.slf4j.Logger;
import com.workfusion.spa.jnative.worker.module.event.EventLogger;
@TaskProcessor(id = "sample-task")
public class SampleTaskProcessor implements ITaskProcessor {
private final BillingInfoService billingInfoService;
@Autowired
public SampleTaskProcessor(...., BillingInfoService billingInfoService) {
// ...
this.billingInfoService = billingInfoService;
}
@Override
public TaskOutputData process(TaskInputData input) throws TaskProcessorException {
String billingInfoJson = // ... build billing info and convert to JSON string
billingInfoSerivce.sendInfo(billingInfoJson);
}
}
Workspace Module is a project that defines all necessary classes and APIs that can be used with Workspace.
To apply the module, do as follows:
Add a dependency to your project's
pom.xml. Note that for projects created using the JNW archetype, this module is already included inpom.xml:<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-workspace</artifactId>
</dependency>Inject the workspace service into the component:
import com.google.common.collect.ImmutableMap;
import com.workfusion.workspace.client.shared.service.WorkspaceClientService;
@TaskProcessor(id = "sample-task")
public class SampleTaskProcessor implements ITaskProcessor {
private final WorkspaceClientService workspaceClientService;
@Autowired
public SampleTaskProcessor(...., WorkspaceClientService workspaceClientService) {
// ...
this.workspaceClientService = workspaceClientService;
}
@Override
public TaskOutputData process(TaskInputData input) {
Map<String, List<String>> params = ImmutableMap.of("country", Arrays.asList("US", "CA"));
PageDtoInternalAssignmentDirectLinkResponse response = workspaceClientService.getAssignments(params, 1, 10);
for (InternalAssignmentDirectLinkResponse assignmentDirectLink : response.getItems()) {
System.out.println("Direct link for assignment: " + assignmentDirectLink.getLink());
}
}
}
Distributed Service Module contains a service for working with the distributed object pool (analog to the WebHarvest pool plugin).
To apply the module, do as follows:
Add a dependency to your project's
pom.xml. Note that for projects created using the JNW archetype, the dependency is already present inpom.xml. In this case, uncomment it.<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-distributed-services</artifactId>
</dependency>Inject the object pool service into your processor.
import com.workfusion.spa.jnative.worker.module.distributed.services.impl.ObjectPoolRunnerFactory;
@TaskProcessor(id = "sample-task")
public class SampleTaskProcessor implements ITaskProcessor {
private final Logger logger;
private final ObjectPoolRunnerFactory objectPoolRunnerFactory;
@Autowired
public SampleTaskProcessor(EventLogger workerLogger, ObjectPoolRunnerFactory objectPoolRunnerFactory) {
this.logger = workerLogger.getEventLogger();
this.objectPoolRunnerFactory = objectPoolRunnerFactory;
}
@Override
public TaskOutputData process(TaskInputData input) throws TaskProcessorException {
try {
//Define a supplier that provides a list of shared objects
List sharedObjects = Arrays.asList("a", "b");
Supplier<List<Object>> sharedObjectSupplier = () -> sharedObjects;
//Define a function to process a shared object
Function<Object, Object> sharedObjectProcessor = o -> {
try {
Thread.sleep(2000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
logger.warn("OBJECT FROM POOL " + o);
doSomethingWithSharedObject(o);
return null;
};
//Create and call objectPoolRunner and pass the shared object supplier and the processor
ObjectPoolRunner objectPoolRunner = objectPoolRunnerFactory.createObjectPoolRunner("test_pool_id");
objectPoolRunner.executeUsingObjectPool(sharedObjectSupplier, sharedObjectProcessor);
//Build output
return new TaskOutputData()
.addRow(new TaskOutputRow()
.put("some_field", "some_value")
.put("other_field", "other_value")
);
} catch (Exception e) {
logger.error(e.getMessage(), e);
throw e;
}
}
You can also mock ObjectPoolProcessor in your test context Worker.
class SampleTaskProcessorTest extends BaseProcessorTest {
@Autowired
TaskProcessingRouter router;
//It's mock
@Autowired
ObjectPoolProcessor objectPoolProcessor;
@DisplayName("Simple Task Processor generated test case")
@Test
void simpleTaskProcessorTest() {
MySerializableObject object = new MySerializableObject();
when(objectPoolProcessor.borrowObject("any-string", Arrays.asList(object))).thenReturn(object);
....
}
}
The modules are added to pom.xml but commented. To use them, uncomment the corresponding lines:
<dependencies>
<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-event</artifactId>
</dependency>
<!--uncomment next dependency, if module for sending billing info is needed-->
<!--<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-billing</artifactId>
</dependency>-->
<!--uncomment next dependency, if a module, which works with client secrets, is needed-->
<!--<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-secrets</artifactId>
</dependency>-->
</dependencies>
You can add other modules manually in the same way as you add any other Maven dependencies:
<dependencies>
<!-- ... -->
<dependency>
<groupId>com.workfusion.spa.java.native.worker</groupId>
<artifactId>module-with-some-functionality</artifactId>
<version>1.2.3</version>
</dependency>
</dependencies>
Check Worker artifact size
During the development process, you might need to add third-party Maven dependencies to implement specific functionality in your JNW. It is crucial to carefully manage this process by adding only essential dependencies and excluding any unnecessary or unused transitive dependencies from those you include.
Keeping the resulting Worker artifact (JAR) as small as possible is vital. A smaller Worker artifact ensures shorter startup time and reduces network, NFS, and CPU load on Master and Agent hosts.
It is strongly recommended to keep the resulting Worker artifact JAR size under 300 MB. Failure to adhere to this recommendation can significantly degrade the Platform's performance and stability under load.
Implement task functionality
To implement the Worker functionality, add the ITaskProcessor classes: one class per each task type.
In the generated application, the archetype creates SampleTaskProcessor in the task sub-package. You can use SampleTaskProcessor as an example or a starter for task processors. This sample task already contains an injected logger for sending events to CT.
@TaskProcessor(id = "sample-task")
public class SampleTaskProcessor implements ITaskProcessor {
//Java Native Worker logger, all messages will be sent into CT as an event.
private final Logger logger;
@Autowired
public SampleTaskProcessor(EventLogger workerLogger) {
this.logger = workerLogger.getEventLogger();
}
@Override
public TaskOutputData process(TaskInputData input) throws TaskProcessorException {
try {
//task processing logic
//send log by worker logger into Control Tower as an event.
logger.info("Some event info for CT side.");
//build output
return new TaskOutputData()
.addRow(new TaskOutputRow()
//.put("some_field", "some_value")
//.put("other_field", "other_value")
)
//add more result rows if required
;
} catch (Exception e) {
logger.error(e.getMessage(), e);
throw e;
}
}
}
A Task processor is a spring component, so you can use any spring mechanism in it, for example:
- Auto-wiring
- Post-construct and per-destroy
- Access environment and configuration, and so on
Build Worker locally
You can build a Worker with the standard maven mvn clean install command. After building it, you get a Spring Boot application artifact in the /target directory. You can deploy the artifact to a target environment as any other Worker's artifacts.
Create BP with Java Worker tasks
To add tasks to the JNW, perform the following steps:
- Create a BP.
- Add a Bot Task and edit its source:
- Add
type="java"to the root<config>tag. - Remove everything inside
<config>and populate the tag with the Java native task metadata.
- Add
- Save the step.
- Save the BP.
The JNW tasks have the minimum structure:
<?xml version="1.0" encoding="UTF-8"?>
<config type="java">
<worker>${GAV}</worker>
<processor>${processorId}</processor>
</config>
Here, replace:
${GAV}with your Worker's currentgroupId:artifactId:version. You can find this parameter inpom.xml.${processordId}with the corresponding task processor ID. See theidattribute in the@TaskProcessorannotation in a corresponding class.
For example, groupId:artifactId:version ($GAV) in your Worker's pom.xml looks as follows:
<project>
<groupId>my.comp</groupId>
<artifactId>my-worker</artifactId>
<version>1.2.3</version>
...
You create two task processors:
@TaskProcessor(id = "first-task-type")
public class FirstTaskProcessor implements ITaskProcessor { ... }
@TaskProcessor(id = "second-task-type")
public class SecondTaskProcessor implements ITaskProcessor { ... }
In this case, the tasks are as follows:
For
FirstTaskProcessor:<?xml version="1.0" encoding="UTF-8"?>
<config type="java">
<worker>my.comp:my-worker:1.2.3</worker>
<processor>first-task-type</processor>
</config>For
SecondTaskProcessor:<?xml version="1.0" encoding="UTF-8"?>
<config type="java">
<worker>my.comp:my-worker:1.2.3</worker>
<processor>second-task-type</processor>
</config>
You can use the following Task parameters:
| Parameter | Required | Description |
|---|---|---|
worker | Yes | groupId:artifactId:version of the JNW artifact. |
processor | Yes | Task processor ID. The code is used to find a processor class and route a task to it. |
splitData | No | Parameter that defines the split-data behavior. Values:
|
sendResultToCaller | No | Parameter that defines if CT must send results to an external connector (for example, REST connector), a caller BP, and so on. Implements the same logic as the send-to-external-connector attribute in export-plugin. Values:
|
Run BP to test logic
Deploy Worker to CT
To test the JNW, deploy it to a target environment and run the corresponding BP. You can deploy the Worker by importing the artifact as a part of the AI Agent's package.
Run Worker from IDE in debug mode
You can run the Worker manually to debug it in IDE. In this case, provide the minimum set of configuration parameters.
Full environment
To run the Worker against a full environment, do the following preparations and preconditions checks:
Make sure the Worker is not deployed to Nexus on the environment to prevent WMS from starting on BEP cluster Worker instances that "steal" tasks from your locally running Worker. If the Worker is already deployed on Nexus, you must perform one of the following actions:
Delete the Worker artifact from Nexus and the shared directory.
Stop WMS and
killcustom Workers (if any) in Marathon. Note that this prevents starting any Worker on a BEP cluster. Thus, if your BP has mixed step types, it will not work.
Download the client certificate
/opt/workfusion/ssl/vault_workfusion.p12for Vault to a local machine.Copy the
secure.storage.*properties from the/opt/workfusion/conf/workfusion.propertiesfile. Add the properties to the Worker configuration parameters. Replace the value forsecure.storage.client.certificatewith a local path to the downloadedvault_workfusion.p12.Run the Worker with the following parameters:
spring.cloud.zookeeper.connect-string=${master-host}:2181
execution.task.queue.input=exec.${ct-client-id}.worker.${gav}.DEFAULT
secure.storage.type=VAULT
# ... and other 'secure.storage.*' parametersRun the BP.
Deploy Worker to Work.AI platform
The JNW, like any other Worker types (CT, AutoML, OCR), must be deployed to Nexus into the workers repository, so WMS can download and start it.
To deploy Java Native Workers to the Work.AI platform, import the Worker artifact as a part of an AI Agent Asset Bundle.
You can import the JNW artifact together with a Bot Config, where it is used, with the help of the Asset Bundle import mechanism. In this case, two components are used:
Import artifact
Before importing the artifact, ensure that you have a JAR file with some groupId:artifactId:version names.
The following instruction is applicable for building a minimum standalone bundle from scratch. The bundle contains only a JNW artifact or for adding Worker artifact to some existing bundle with repackage (steps 3, 4, 5, 7).
To import the Worker artifact, perform the following steps:
Create the
artifactory-dependencydirectory in the bundle if it does not exist.In the
artifactory-dependencydirectory, create theworkersdirectory as the first underlying directory name. It represents the name of the Nexus repository where the artifact must be imported.In the
workersdirectory, create the tree of$GAVdirectories. Thegroupname must be split into the hierarchy of sub-directories by a dot in thegroupname, for example,com.supertest.In the
versiondirectory, place the JAR artifact named according to the convention:artifact_name-version.jar.In the bundle's root directory (the same layer as the
artifactory-dependencydirectory), create themeta-info.jsonfile. See the structure and description in Package assets into AI Agent Asset Bundle.artifactory-dependency
│ └── workers
│ │ └── com
│ │ ├── supertest
│ │ └── test-native-worker
│ │ └── 1.0
│ │ └── test-native-worker-1.0.jarIn this case, Worker's
GAViscom.supertest:test-native-worker:1.0.Archive the above structure into a zip archive. Your bundle is now ready.
The workers Nexus repository is a release repository. You can't import an artifact containing the SNAPSHOT postfix in its version. The release artifact versions are required.
Import Bot Config
When importing a Bot Config, you have the following options:
- Import from the JNW artifact.
- Import from an Asset Bundle.
Import from Java Native Worker artifact
A Bot Config that contains the JNW processor call can be imported directly from the JNW artifact.
Place the Bot config XMLs in the src/main/resources/configs/main directory as shown below:
my-custom-worker
└──src
└──main
└──resources
└──configs
└──main
├──email_intake.xml
└──validation.xml
When you build the JNW and import it as a part of the Asset Bundle, machine configs are also imported to CT.
You can only import machine configs that call to the JNW processors in such a way. This approach makes importing WebHarvest or some arbitrary Bot Config Bundle impossible.
Import from Asset Bundle
A Bot Config (step) configured to use the JNW processing can be imported similarly to any other.
Before importing a Bot Config from the Asset Bundle, ensure you have the bot-config.xml file exported previously from another instance.
To import a Bot Config, do as follows:
Create the
bot-configdirectory if it doesn't exist in a bundle.Place the XML bot config file in the
bot-configdirectory.Create the
meta-info.jsonfile as described in the previous section.Archive the above files and directories into the ZIP archive. Your bundle is now ready.
bot-config
│ └── some-config.xml
See the example in some-config.xml.
The config contains UUIDs, names, and other components taken from the original instance. If you manually modify its machine configuration in a text editor or export the same step with different machine code configurations without changing UUIDs, the bot config machine code will be overwritten during the import if an existing configuration is present with identical IDs.
The bundle structure with the JNW and a bot configuration looks as follows:
bot-config
│ └── some-config.xml
artifactory-dependency
│ └── workers
│ │ └── com
│ │ ├── supertest
│ │ └── test-native-worker
│ │ └── 1.0
│ │ └── test-native-worker-1.0.jar
Select Java version
WMS supports running BEP Workers on different versions of JDK. Currently, JDK versions 8 and 17 are supported. If the version is not defined, JDK 8 is used.
To set a target JDK version, add the following lines to the META-INF/worker/worker.yml file:
command: ...
app-name: ...
// ... other configuration parameters
java-version: 17
java-version must be a high-level parameter without any indent in YAML.