Version: 10.3.2

Generate Worker application from archetype

Overview

The best way to get started with Java Native Worker (JNW) is to use an archetype to create an AI Agent project with all the needed dependencies and a preliminary structure. There are two archetypes you can use depending on the case or project in which you want to integrate the JNW approach:

java-native-worker-archetype: a single-module archetype
java-native-worker-bundle-archetype: a multi-module archetype

tip

For the JNW compatibility matrix, see the Java Native Workers guide.

The article dwells on how to generate and configure (using archetype properties) projects based on the archetypes, integrate a JNW module into an existing project, connect additional modules, and get started with a new project or module.

Before generating anything from an archetype, choose the one that will be most useful in your particular case. The following table shows the pros and cons of each archetype.

Feature	`java-native-worker-archetype`	`java-native-worker-bundle-archetype`
Can be integrated into an existing project without any special changes	✓	✘
Provides a JNW JAR	✓	✓
Provides a zip archive of the AI Agent bundle	✘	✓
Provides a module for converting schemas into data models	✘	✓
Provides integration tests	✘	✓

The main difference between the archetypes is that java-native-worker-archetype is a single module archetype you can use as a module or a standalone project. This archetype is simple, containing only the needed dependencies and nothing more. At the same time, java-native-worker-bundle-archetype is a multi-module archetype with everything needed to create an AI Agent project based on the JNW approach. Thus, java-native-worker-bundle-archetype is a JNW with peripheral modules such as a schema to the Java data model converter, a module for creating an AI Agent bundle, and a module for integration tests, while java-native-worker-archetype is just a JNW module without any additional elements.

If you only need the JNW JAR or you need to add a JNW module to an existing project, we recommend using java-native-worker-archetype. If you need to create a complete AI Agent cycle, use java-native-worker-bundle-archetype.

Before you generate a Worker application, you should prepare the environment. For instructions, see Prepare environment for archetype generation.

java-native-worker-archetype

java-native-worker-archetype is a single-module archetype you can use as a module within an existing multi-module project or as a separate project.

Generate project from java-native-worker-archetype

The generation process looks like a standard Maven archetype generation procedure:

To generate a Worker application from the archetype, run the following command in the parent directory:

mvn archetype:generate -DarchetypeGroupId=com.workfusion.spa.core -DarchetypeArtifactId=java-native-worker-archetype -DarchetypeVersion=X.X.X.X

Provide the following properties when the command prompt appears:

groupId: Worker's Maven group ID.
artifactId: Worker's Maven artifact ID. Also, it is the default directory and the target artifact name.
version: Worker's Maven version.
package: root package for Worker Java classes.

Define value for property 'groupId': com.mycompany
Define value for property 'artifactId': my-test-worker
Define value for property 'version' 1.0-SNAPSHOT: :
Define value for property 'package' com.mycompany: : com.mycompany.worker
Confirm properties configuration:
groupId: com.mycompany
artifactId: my-test-worker
version: 1.0-SNAPSHOT
package: com.mycompany.worker
Y: : y

A new project appears in a sub-directory with the name taken from the artifactId value. The new project is the pre-configured Spring Boot application with minimum dependencies required to implement the base Worker logic and a sample task processor.

Integrate java-native-worker-archetype to existing multi-module project

To integrate the JNW module into an existing project, repeat the steps from the Generate project from java-native-worker-archetype guide from the existing project directory, add your project's root module as a parent to Worker's pom.xml, and add generated artifactId to modules in the root pom.xml file.

warning

Be careful with dependency versions in the Worker module since the Worker has its own dependentManagement section.

good practice

Move the dependencyManagement section from the Worker module to the root of pom.xml. This refactoring allows you to manage dependencies throughout the project and avoid facing version conflicts in the future.

Manage dependencies

The dependencyManagement section contains the import of BOM with all JNW dependency versions that you might need in the development or testing process. To start using additional JNW modules, add the dependency without the version (the version is in the BOMs) in the dependencies section. You can add the third-party dependencies as in a simple Maven project.

View generated module structure

The structure of the generated module is as follows:

Worker application code base:
- JavaWorkerApplication.java: Spring Boot application starter class
- Tasks processors:
  - SampleTaskProcessor.java: simple example usage of TaskProcessor that does not support contracts
  - SampleSchemaBasedTaskProcessor.java: simple example usage of JsonTaskProcessor that supports and uses schemas and contracts
Worker resources:
- Configs:
  - sample-json-task.xml: SampleTaskProcessor step source code.
  - sample-task.xml: SampleSchemaBasedTaskProcessor step source code.
  - sample-json-task.in-contract.yaml: input contract for the SampleSchemaBasedTaskProcessor processor.
  - sample-json-task.out-contract.yaml: output contract for the SampleSchemaBasedTaskProcessor processor.
- worker.yml: contains elasticapm Worker configuration, namely cpu, heap, and so on.
- application.yml: contains the JNW application configuration (Spring-based configuration with WorkFusion specific).
- logback.xml: log configuration to control the log level.
Worker TaskProcessors tests based on the worker-task-test library:
- SampleTaskProcessorTest.java: test for the SampleTaskProcessor processor.
- SampleSchemaBasedTaskProcessorTest.java: test for the SampleSchemaBasedTaskProcessor processor.

java-native-worker-bundle-archetype

java-native-worker-bundle-archetype is a complete and ready-to-use archetype for an AI Agent based on the JNW approach. Unlike java-native-worker-archetype, the archetype uses jnw-toolkit to simplify working with processors and contracts, namely generating contracts, XML processors based on method parameters and return type.

Generate project from java-native-worker-bundle-archetype

The generation process looks like a standard Maven archetype generation procedure:

To generate an AI Agent JNW project from the archetype, run the following command in the parent directory:

mvn archetype:generate -DarchetypeGroupId=com.workfusion.spa.core -DarchetypeArtifactId=java-native-worker-bundle-archetype -DarchetypeVersion=X.X.X.X

Provide the following properties when the command prompt appears:
Standard Maven archetype parameters:
- groupId: group ID that identifies your project across all projects. The group ID should follow Java's package name rules. Thus, it starts with a reversed domain name you control, for example, com.mycompany.mytest.
- artifactId: name of the JAR without a version in Maven archetypes. If you create it, you can choose any name with lowercase letters and no special characters. In our case, artifactId is used to name the folder where your project is generated. Also, it is used as a prefix in the names of all generated modules. For example, if artifactId is set to my-test-digital-worker, the following modules are generated:
  - my-test-digital-worker-model
  - my-test-digital-worker-worker
  - my-test-digital-worker-package
  - my-test-digital-worker-test
- version: Maven's version of the project to be created. Do not use SNAPSHOT or dates in it. Follow simple versioning patterns like 1, 5.3, 8.4.5. The Maven version is used to define an AI Agent version.
- package: Java package to be used in the project. By default, Maven suggests the same value as in groupId. It is used as a package path (split by points) in your project. Define it like com.mycompany.mytest.
AI Agent parameters:
- usecase-name: AI Agent name to be associated with the project and used as a human-readable name. Use upper and lower case Latin symbols, spaces, and digits—this is essentially a string for representation. Keep it relatively short (up to 32 characters) to avoid breaking markup on the UI, for example, KYC Latin America.
- usecase-code: AI Agent code to be associated with the project. It is used as a technical ID of the AI Agent, for example, as a part of the name for each Data Store created for this AI Agent. Use only Latin lowercase symbols without spaces, for example, inv, star, kmk, and so on.
- schema-version: version of the generated example schema. By default, the version is 1.0.0.
Project repositories and dependencies management parameters:
- control-tower-url: URL of the remote Control Tower instance where the AI Agent is deployed for testing. The default value is https://instance.workfusion.com. You can easily change it in pom.xml later.
- nexus-url: URL of the Nexus server. The default value is http://localhost:18081 that is the Nexus server from the local Work.AI Developer. You can easily change it in pom.xml later.
- Parameters that allow you to predefine additional module dependencies for JNW (all of these options are false by default) to be added to the Maven -worker module:
  - enable-all-jnw-modules: if true, all additional modules for the JNW core are added, namely module-billing, module-secrets, and so on.
  - enable-all-jnw-toolkit-modules: if true, all additional modules for the JNW toolkit are added, namely jnw-toolkit-configuration, jnw-toolkit-datastores, and so on.
  - enable-all-worker-task-test-modules: if true, all additional modules for the JNW testing library are added, namely worker-task-test-orm-lite-module, worker-task-test-s3-module, and so on.
    Define value for property 'control-tower-url' https://instance.workfusion.com: : Define value for property 'nexus-url' http://localhost:18081: : Define value for property 'usecase-name': test-name Define value for property 'usecase-code': test-code Define value for property 'schema-version' 1.0.0: : Define value for property 'enable-all-jnw-modules' false: : Define value for property 'enable-all-jnw-toolkit-modules' false: : Define value for property 'enable-all-worker-task-test-modules' false: : Define value for property 'groupId': com.mycompany.mytest Define value for property 'artifactId': my-test-digital-worker Define value for property 'version' 1.0-SNAPSHOT: : 1.0.0 Define value for property 'package' com.mycompany.mytest: : Confirm properties configuration: control-tower-url: https://instance.workfusion.com nexus-url: http://localhost:18081 usecase-name: test-name usecase-code: test-code schema-version: 1.0.0 enable-all-jnw-modules: false enable-all-jnw-toolkit-modules: false enable-all-worker-task-test-modules: false groupId: com.mycompany.mytest artifactId: my-test-digital-worker version: 1.0.0 package: com.mycompany.mytest
Once you set all properties, confirm your choice:
- If all looks good, type Y for Maven to start building a project.
- If something is set incorrectly and you want to get back to the configuration, type N to get back to the properties setup.

A new project appears in a sub-directory with the name taken from the artifactId value. The new multi-module AI Agent project contains all the necessary modules for preparing and importing the AI Agent into an environment and running integration tests from the project in a real environment. The artifactId-worker module is the core of your AI Agent, where you add new processors provided by your AI Agent.

Manage dependencies

The dependency management procedure is similar to the dependency management in java-native-worker-archetype but with a few minor changes. The artefactId-worker module is where you mostly need new dependencies but the main dependentManagement section is located in the root pom.xml file of the project. Thus, it is recommended to add versions of new dependencies to root pom.xml. In the worker, you can add them to the dependencies section.

View generated project structure

The generated project contains four modules and a project pom.xml file. The root pom.xml file contains the dependencyManagement and pluginManagement sections, information about local and remote profiles required to import the AI Agent bundle, and repositories.

List of modules:

artifactId-model: schema to the Java data model converter
artifactId-worker: JNW application
artifactId-package: AI Agent bundle
artifactId-test: Spoke tests

Model

The model module generates a Java data model from the schema.yaml file stored in the resources folder of the module. For this, openapi-generator-maven-plugin is used. The result of packaging this module is a JAR file containing standard Java classes that describe the OpenAPI model you provide.

To use the models, add the module as a dependency to your Worker or other project or module.

By default, in a project created from an archetype, the module contains a schema for an example process. When you replace it with your own schema, be careful with the filename because the plugin configuration is set to the content that is actually stored in the schema.yaml file in the resources folder.

tip

After packaging, you can find Java classes by the artefactId/artefactId-model/target/classes/com/mycompany/worker/model path.

Worker

The worker module is the main module of the project that stores your TaskProcessors and related Java code.

note

JNW is a Spring application. You can use most of the Spring features inside your Worker, namely services, bean management, and so on.

The module comes with several pre-generated files:

Worker application code base:
- JavaWorkerApplication.java: Spring Boot application starter class
- Tasks provided in the jnw-toolkit view:
  - DocumentProcessingTasks.java: contains an implementation of the DocumentProvider and DocumentProcessor processors with the jnw-toolkit annotations.
  - NotificationTasks.java: contains an implementation of the SenderNotification processor with the jnw-toolkit annotations.
  - All processors represent a chain of tasks in which you first receive and process documents and then notify the user about the processing result.
- Simple service examples:
  - DocumentService: processes Document, where Document is a model from the model module.
  - EmailService: reads a document from an email in the correct format.
  - NotificationService: sends notifications to the user.
- Email.java: a data model that contains the Document and Sender data and allows the correct output contract to be generated for the processors.
Worker resources:
- worker.yml: contains the elasticapm Worker configuration, namely cpu, heap, and so on.
- application.yml: contains the JNW application configuration (Spring-based configuration with WorkFusion specific).
- logback.xml: log configuration to control the log level.
Worker TaskProcessors tests based on worker-task-test library:
- DocumentProcessorTest.java: test for the DocumentProcessor processor
- DocumentProviderTest.java: test for the DocumentProvider processor
- SenderNotificationTest.java: test for the SenderNotification processor

note

DocumentProvider.java, DocumentProcessor.java, and SenderNotification.java are generated with jnw-toolkit.

Package

The package module or the Asset Bundle module comprises a Business Process (BP) and its supporting resources, such as a Worker and a schema.

In this case, the package module already contains a BP example, which is Example_BP.zip. Also, in the assembly folder, there is package.xml used as a descriptor for copying dynamic resources.

Let's describe each folder inside the package module.

/artifactory-dependency

The directory is required if a JNW is used. Otherwise, it is optional. The directory's content maps to zero, one, or multiple JNWs.

In most cases, there is no need to add assets to the artifactory-dependency directory. After you run mvn clean package, JNW artifact files from the worker module of the same project are deployed automatically to the target/artifactory-dependency folder.

/schema

The directory is required if schemas are used. Otherwise, it is optional.

The archetype schema is populated with the schema.yaml file from the artifactId-model JAR.

warning

When configuring the assembly plugin, the artifactId-model JAR is unpacked and all **/*.yaml files are moved to the /schema folder. Be careful with the files you add to the artifacId-model module.

/business-process

Pack the directory with BP packages created as zip files in the server Control Tower in the development environment.

To add a new BP, do as follows:

Create a BP in the server or local Control Tower in your development environment.
Export the BP from Control Tower.
Put the exported BP into the resources/business-process folder.

info

The system automatically creates a BP draft for every imported BP package in Control Tower.

/datastore

The datastore directory serves as a storage for any content essential for creating and maintaining BPs. It is also the most convenient way to share temporary data across bot steps.

Pack the directory with CSV files to be used for creating Data Stores. One CSV file corresponds to one Data Store. The names of CSV files map to the names of Data Stores.

The data format in the files must be the same as if you uploaded data to a Data Store via Control Tower. Each file must contain headers and at least one record.

Mind that Data Stores based on the files contain a single data type, which is NVarchar(max).

For details, refer to Manage Data Model.

/s3

Pack the directory with static resources if you need them for your AI Agent. Each top-level sub-directory in the s3 directory corresponds to an S3 bucket. Therefore, the names of the top-level sub-directories in the s3 directory must be mapped to the names of the S3 buckets.

meta-info.json

The file is required and must be a valid JSON file. The NAME and WF_VERSION parameter pairs are required.

The USE_CASE_CODE and USE_CASE_VERSION parameters are optional but recommended since they allow viewing AI Agent versions on the Use Cases list. They define the AI Agent version where the imported bundle is applied. All AI Agent assets are to be tracked within the version. If the parameters are used, they must be used together.

The following restrictions apply to the parameters:

USE_CASE_CODE can contain letters in uppercase, dashes (-), underlines (_), and digits.
USE_CASE_VERSION must contain a semantic version of an AI Agent, for example, 1.0.2, 2.3.5-SNAPSHOT. For details, see Semantic Versioning.

If an Asset Bundle is structured correctly, it contains all the components needed for BPs to run in a target (production) environment. However, any component from the structure above is optional, except for meta-info.json.

Test

The Test module is intended to store integration and acceptance tests.

The module comes with several pre-generated files:

ExampleBusinessProcessIT.java: Spoke test example for the Example_BP.zip BP from the package module
ProjectProperties.java: helper class to read data from the project.properties file
project.properties: project-related settings to ease Spoke tests creation and usage

note

If you are unaware of the Spoke library and how to test BPs, refer to the Spoke tutorial and the Business Process integration testing guide.

Here is a quick overview of what ExampleBusinessProcessIT does on execution:

Uploads the project bundle from the package module to a Control Tower instance.
Runs the Example_BP.zip BP.
Waits until the BP is completed successfully.
Asserts that the output data contains a required and processed processedDocument entry.

By default, all Spoke tests from the Test module are tied to the Maven verify phase and activated by the run-spoke-tests profile. To run the tests during a Maven build, use the following command:

mvn clean verify -Prun-spoke-tests

Before running a Spoke test, make sure to prepare the spoke.configuration or spoke-it.properties file with the target Control Tower settings. Follow the Spoke guide to get more information about Spoke configuration.

View example Business Process

An sample BP is added to the archetype to show how the package module works and to allow the user to understand how the data context within a diagram-based BP works. By data context, we mean that you can access some data from the already executed processors in other processors, regardless of whether the data was in the previous processor's output contract.

The BP contains only three processors connected step-by-step:

Document processors:

DocumentProvider (multi-row output): reads emails and provides them as documents and sender information.
- Input contract:
- Output contract:
  - Documents
  - Sender
DocumentProcessor (single-row output): processes the document according to the rules provided by the service.
- Input contract:
  - Documents
- Output contract:
  - ProcessedDocument

DocumentProcessingTasks

@Component
public class DocumentProcessingTasks {

    private final DocumentService documentService;
    private final EmailService emailService;

    public DocumentProcessingTasks(EmailService emailService, DocumentService documentService) {
        this.emailService = emailService;
        this.documentService = documentService;
    }

    @AutoContract
    @AutoTaskProcessor(className = "DocumentProvider")
    public List<Email> documentProvider() {
        return emailService.readEmails();
    }

    @AutoContract
    @AutoTaskProcessor(className = "DocumentProcessor")
    @TaskProcessorOutput(columnName = "processedDocument", deconstruct = false)
    public List<ProcessedDocument> processDocuments(Documents documents) {
        return documents.getDocuments().stream().map(documentService::processDocument).collect(Collectors.toList());
    }

}

Notification:

SenderNotification (single-row output): notifies the sender whether the sent document is processed or not.
- Input contract:
  - Sender
  - ProcessedDocument
- Output contract:
  - Sender

NotificationTasks

@Component
public class NotificationTasks {

    private final NotificationService notificationService;

    public NotificationTasks(NotificationService notificationService) {
        this.notificationService = notificationService;
    }

    @AutoContract
    @AutoTaskProcessor(className = "SenderNotification")
    @TaskProcessorOutput(columnName = "sender", deconstruct = false)
    public Sender senderNotification(Sender sender, ProcessedDocument processedDocument) {
        notificationService.notifySenderAboutProcessedDocument(sender, processedDocument);
        return sender;
    }

}

note

The archetype uses a mock service implementation and is added for demonstration purposes only. In real cases, this type of implementation can be moved into the testing area and annotated with a Spring test profile.

As you can see, Sender appears in the first step. The data is not needed in the next step but is expected in the last one. Using a schema-based data context, you don't have to move data from task to task; you can simply call it on the right processor.

Overview​

java-native-worker-archetype​

Generate project from java-native-worker-archetype​

Integrate java-native-worker-archetype to existing multi-module project​

Manage dependencies​

View generated module structure​

java-native-worker-bundle-archetype​

Generate project from java-native-worker-bundle-archetype​

Manage dependencies​

View generated project structure​

Model​

Worker​

Package​

/artifactory-dependency​

/schema​

/business-process​

/datastore​

/s3​

meta-info.json​

Test​

View example Business Process​

Overview

java-native-worker-archetype

Generate project from java-native-worker-archetype

Integrate java-native-worker-archetype to existing multi-module project

Manage dependencies

View generated module structure

java-native-worker-bundle-archetype

Generate project from java-native-worker-bundle-archetype

Manage dependencies

View generated project structure

Model

Worker

Package

/artifactory-dependency

/schema

/business-process

/datastore

/s3

meta-info.json

Test

View example Business Process