Edit

View BEP and AutoML integration capabilities

The main goal of AutoML and BEP integration is to use the BEP platform for:

• AutoML execution
• AutoML training

AutoML execution

In pre-BEP, all model execution requests were synchronous using REST API from the corresponding bot steps. This approach still works with BEP but has performance issues like:

• The limited number of simultaneous HTTP connections for a REST request.
• A lot of workers waiting for AutoML execution leading to cluster resources under utilization. In the worst case, WMS kills Control Tower (CT) workers waiting for a response to start an AutoML worker to process all these requests.

As CT should not be tightly coupled with the AutoML model input data format, the Bridge mechanism inside the AutoML Service is introduced. CT sends an AutoML execution request as a CT task for AutoML Bridge to consume the task, convert it to an AutoML request, and send to an AutoML worker. When the worker processes the task and sends results, the AutoML Bridge performs the same flow for results: consumes them, converts to CT results, and sends to CT.

For details on AutoML model execution with the Bridge framework, see Execute AutoML Model: Bridge.

The overall AutoML flow is as follows.

1. An Execute AutoML Model: Bridge step is introduced. It is like a bot step that contains special XML with tags for model execution parameters instead of the web-harvest DSL XML.
2. When CT sees the step, it sends a special task to the AutoML Bridge channel, a special queue managed by the Bridge component inside the AutoML Service.
3. The Bridge converts incoming CT tasks into AutoML tasks and sends them to AutoML Execution input queues.
4. The tasks are brokered to AutoML worker queues by the standard TDS mechanism. AutoML execution workers are also managed by standard WMS.
5. When an AutoML worker receives the task, it executes the model and sends the AutoML execution result to a special AutoML result channel.
6. The Bridge component consumes the result message and converts it to a CT result message sent to the standard CT result queue.
7. When CT gets the result, it processes the message similarly to a standard CT task result (a complete record on the AutoML execution step) and sends it further to the Business Process (BP) flow.

AutoML training

In the pre-BEP implementation, the AutoML training is governed by the root step. A worker receiving a training task (a root task), performs the AutoML training orchestration. With BEP, this approach is inappropriate as any worker can crash or be stopped by WMS. Due to this, the training process orchestration is moved to the AutoML Service.

When the AutoML Service receives the training task, it generates a BPMN file for the given model and executes it using the local storage for the BP persisting state. For each step in the BP, the service sends a task to AutoML workers using the standard BEP mechanism.

The overall BP training execution from the CT perspective has the same flow as before.

1. The training BP gets to the Start Training step containing the automl-train-start plugin.
2. automl-train-start makes the startTrain REST call to the AutoML Gateway Service.
3. The AutoML Gateway Service forwards a request to the AutoML Service that starts a training process asynchronously.
4. The AutoML Service returns the response with trainingId to the AutoML Gateway Service that delivers the response to CT.
5. CT completes the Start Training step, checks a training status by executing periodically, and calls the automl-train-status plugin that makes a REST call to the gateway.
6. The call is forwarded to the AutoML Service that checks the BP status internally as it governs the training orchestration and returns the actual status of the training BP:
   • If the status is in progress, the check status step is exited using the release plugin. The worker can process other tasks if needed.
   • If the status is completed, the BP proceeds to the next step.