Get started with AutoML SDK

The article features a high-level overview of how you can develop a machine learning (ML) model with AutoML SDK and publish it to Control Tower (CT) for further use in a Business Process (BP). The general flow is shown in the figure below:

Create Maven project

Using an AutoML Maven Archetype, create an AutoML SDK project as described below:

1. Make sure you have access to the repository with AutoML Archetypes and dependencies. Choose between the two options:

   • Use the public WorkFusion Nexus repository server, if allowed within your organization. To access repositories, you need WorkFusion credentials.
   • Use your company’s on-premise repository server that is a part of every WorkFusion platform installation and is typically used from the DEV instance.

2. Configure a local environment as detailed in the guide.

3. Start a new Maven project from an AutoML Archetype as described in the Start from Archetype article.

Implement model

Now that you have a ready-to-go AutoML SDK project, implement a generic model as described below:

1. Configure ModelTrainingRunner to extract required fields in accordance with the Train model locally guide.
2. Run ModelTrainingRunner.
3. Check the model's statistics in the output/model-results-stats folder.

Improve trained model

In some cases, you may need to improve one or more fields in an already existing trained model. Retraining the entire model is a time-consuming operation. To save time, you can improve only the required field and replace the artifact folder containing the old field with a new one containing updates.

To improve a model, follow the steps below:

1. Make a separate copy of your AutoML SDK project.

2. Import one or more configurations to the AutoML SDK configuration file for all fields. In this case, AutoML skips smart selection of components, which significantly reduces the time of further re-training.

3. Comment all fields in your ModelTrainingRunner, uncommenting the ones that need to be improved.

   tip

   • If a model extracts fields but they are formatted incorrectly, consider adding Post-Processors to your AutoML SDK configuration.
   • If a model extracts wrong fields, consider adding Feature Extractors and Annotators to your AutoML SDK configuration.

4. Run ModelTrainingRunner with the updated AutoML SDK configuration.

Reuse existing model

The development flow reusing an existing model is almost identical to that when a model is created. Instead of configuring a new project, import an existing one.

To start developing based on an existing model, follow the steps below:

1. Make sure you have access to the repository with AutoML Archetypes and dependencies. Choose between the two options:

   • Use the public WorkFusion Nexus repository server, if allowed within your organization. To access all repositories, you need WorkFusion credentials.
   • Use your company’s on-premise repository server that is a part of every WorkFusion platform installation and is typically used from the DEV instance.

2. Configure a local environment as detailed in the guide.

3. Import your existing AutoML SDK project to IDEA IDE or Work.AI Developer (Eclipse).

Then, use the model as is or make improvements.

Publish model

After you successfully implement a new model or reuse an already existing one, your next step is to publish it to production. This means you need to build a bundle containing the model and deploy it to a platform instance.

There are two options to publish your model:

• Publish model configuration only. In this case, training is performed on a remote Control Tower (CT) instance. After that, your model becomes available for execution.

• Publish a trained model with all training artifacts. This scenario allows you to avoid remote model retraining and use the model for extraction or classification immediately.

info

The AutoML menu becomes visible only if you have the Manage models permission in CT. For instructions on setting the permission, refer to the Manage Control Tower roles and permissions article.

Publish model configuration

Under this scenario, all you have to do is to build a bundle and deploy it to CT. No local model training or manual deletion of generated model artifacts is required.

To publish a model configuration to CT, follow the steps below:

1. Run maven clean install using the pom.xml file in the root of the project to generate the /deploy folder.

2. Run mvn deploy using the pom.xml file in the [PROJECT]-package module of the project. The target instance URL is defined in the remote profile of your root pom.xml. The credentials are in your Maven’s settings.xml.

   note

   Mind the import logs in the console. If the configuration is correct, you see a success message once the model is deployed. Otherwise, analyze and fix an error.

3. Check the model availability in CT in one of the following ways:

   • Via AutoML UI by checking the Pipelines list.

   • Via AutoML SDK API by sending a listHyperModels request.

tip

Alternatively, you can import a model configuration via the AutoML UI in CT. For details, refer to the Manage pipelines | Import topic.

Mind that the AutoML menu becomes visible only if you have the Manage models permission in CT. For instructions on setting the permission, refer to the Manage Control Tower roles and permissions article.

Publish trained model

Under this scenario, you need to first train and test your model locally and then build a bundle and deploy it to CT. Follow the steps below:

1. Train the model locally as described in the Implement model section.
2. Publish the resulting model configuration as described in the section above.

tip

Alternatively, you can import a model configuration with training artifacts via the AutoML UI in CT. For details, refer to the Manage models | Import topic.

Mind that the AutoML menu becomes visible only if you have the Manage models permission in CT. For instructions on setting the permission, refer to the Manage Control Tower roles and permissions article.

Execute model

info

Large pipelines and trained models exceeding 500 MB can cause high IOPS (high iowait) when AutoML BEP Workers are launched.

To execute an AutoML model in Control Tower, a special Model step is used.

• The step is implemented over ETL Bot Config and shouldn't be changed.
• The step processes based on the one document by one model principle.
• First you collect correct data, send it to the step, and get results—that is the internal CT logic that you cannot influence.
• Bot sources are not supported.
• Stateless execution is supported.

For more information, go to the Add and configure Model, LLM, and Dataset steps guide.