Create automation Manual Task
WorkFusion AutoML empowers intelligent automation by enabling machine learning models to learn from user answers in Manual Tasks (MT).
Automation Manual Tasks are created and designed in Control Tower. After that, they are completed by users in WorkSpace and submitted back to Control Tower. After a model is trained on the submitted data, the WorkFusion platform creates a Cognitive Bot and, subsequently, routes all work to it.
The basic steps to create an automation Manual Task are as follows:
- Create a Manual Task from a proper Operation.
- Prepare and upload input data.
- Configure a model.
Create task from Operation
To set up an automation Manual Task, follow the steps below:
Click Manual Tasks > Create Manual Task.
Select the Operation category and Operation as illustrated below and click Select This Operation.
Upload input data
Prepare the data as described in the Split training and test sets guide. Note that your input file should contain at least 20 records (documents) or more.
To upload data for a model to train on, follow the instruction.
Configure AutoML
note
When designing the automation Manual Task, make sure that all fields that need to be automated by the model are configured as required. Otherwise, the AutoQC process fails to pick those fields.
To train a model and eventually generate a Cognitive Bot, link your Manual Task to AutoML and configure the model settings. For that, go to the Configure AutoML tab of the newly created task.
On the page, you have three options:
If you don't want to automate your Manual Task, select Model not applied.
Select the Execute model option. For details, refer to Developing with AutoML SDK | Execute model.
Use the Train new model option as described below.
Train new model
To configure a model and link it to the created Manual Task, the exact steps are as follows:
On the Configure AutoML tab, select Train new model.
In the Machine Learning Pipelines section, select the Machine Learning Model that corresponds to your task (for example, Information Extraction, Classification, Financial Reports Data Extraction, and so on). This model will learn from human answers and automate your Manual Task by creating a Cognitive Bot.
Select a Cognitive Bot Workflow. The Cognitive Bot is created from the workflow when your Manual Task is completed. To edit the workflow, select the icon next to the Cognitive Bot Workflow drop-down menu.
Select an Automation Training Set.
If you have historical data for training the ML models, upload the data to the Automation Training Sets and then use it for Manual Task automation. When assigning an existing training set to a new Manual Task or doing a quick data assessment for ML purposes, you can view the training set answer availability and its percentage using the Show Data Completeness button.
If you do not have a training set, select the Create new Training Set automatically option.
It is the default option. A new training set is created for storing workers' answers and training ML models to automate your task.
Optionally, click the three-dot button to perform the following actions with an already added training set:
- View
- Rename
- Copy
- Reset Data Status: sets the train_status column for all records to NEW. Used to include all records from a particular training set in ML model training.
Set training configuration
The training configuration flow depends on which Search Engine version is supported by the configured model:
Search Engine 2.0 training configuration
Models with SE 2.0 are already optimized to produce the best results. So, you can adjust only the number of Fields Trained In Parallel. The maximum number of fields trained in parallel is calculated based on the AutoML cluster capacity. The Training Configuration section also gives an estimate of the complete training time for the model.
Search Engine 1.0 training configuration
If the selected Machine Learning Model supports Search Engine 1.0, the Training Configuration section is enabled. By default, you can see preset options for the Training Configuration and the time needed to complete the training for all fields of the original Manual Task.
Move the slider to set up your training configuration according to the complexity of the task. As you move it, the estimated training time is recalculated instantly.
To have more control over the training setup, click Show Details. In the following picture, you can see an example of the Training Configuration section for an Information Extraction ML model.
The following parameters are available:
Max Training Time: maximum time AutoML uses to find the best subset of components required for a single field.
Fields Trained in Parallel: maximum number of fields (sub-models) that can be trained in parallel on an AutoML cluster. By default, the slider is set to the optimal value based on the available cluster capacity.
note
The Fields Trained in Parallel parameter is available only for Information Extraction models and does not apply to Classification models.
Metric: used to compare the results of experiments and select the best one. Available metrics are F1, Precision, or Recall.
The values for the Max Training Time and Metric parameters are predefined and listed in the table below. In contrast, the values for the Fields Trained In Parallel parameter are calculated based on the AutoML cluster capacity. There are four possible preset values ranged from High speed to High quality.
| Slider Position (Preset) | Max Training Time | Metric |
|---|---|---|
| 1—Low Complexity | 1 hour | F1 |
| 2 | 2 hours | F1 |
| 3 | 4 hours | F1 |
| 4—High Complexity | 10 hours | Precision |
You can select one of the preset options, and the option is applied to all fields (sub-models) during training. In case you cannot find an appropriate combination of parameters from the preset options, you can provide custom parameters by clicking the Set Custom Goal link.
Using the mode, you can set up all parameters manually. The parameters have the following restrictions:
- Max Training Time ranges from 1 hour to 10 hours.
- Fields Trained In Parallel is limited by the AutoML cluster capacity.
note
While calculating Fields Trained In Parallel and Complete Training Time, currently running model trainings on an AutoML cluster are not taken into account. In case one or several models are trained on an AutoML cluster, the values for Fields Trained In Parallel and Complete Training Time remain the same and do not reflect the time in the queue. Before setting the Training Configuration parameters, check other models with HPO trained on the same cluster.
Define setting priority
Since there are several places for configuration of model training settings, there are different priorities that each place has. The priorities are listed in the descending order: at the top are the settings applied in the first turn; in the bottom are settings applied last.
For Max Training Time:
- Settings in the Training Configuration section on the Configure AutoML tab
- Settings in ZooKeeper
- Settings in the model configuration
For Fields Trained in Parallel:
- Settings in the Training Configuration section on the Configure AutoML tab
- Settings in ZooKeeper
For Metric:
- Settings in the Training Configuration section on the Configure AutoML tab
- Settings in the model configuration
Note that managing the Training Configuration section requires the Manage Platform Monitor permissions.
Automate Manual Tasks in BP
Besides automating stand-alone Manual Tasks as described above, you can also automate Manual Tasks that are steps in a Business Process (BP).
To automate a Manual Task in a BP, act as follows:
- Set AutoML parameters for the step as described in the Configure AutoML section.
- Run the BP and wait until the Manual Task is completed by human workers in WorkSpace. Model training starts as a background process.
When, in the Manual Tasks list, the Manual Task has the AUTOMATION AVAILABLE label, proceed as below:
Open the Manual Task and go to the AutoML tab.
Select the minimal acceptable accuracy and click Apply Recommendation. For details, refer to Retrain IE Cognitive Bot model.
The Manual Task on the original BP diagram is substituted with an Automation Sub-Process for the step.
note
The original BP stays unchanged. The WorkFusion engine creates a new process definition with links to the same steps as in the original, except that the automated Manual Task is substituted with an Automation BP.
Upload a new data batch for extraction and launch the new BP. You can expand the sub-process and view the automation progress.
tip
You can automate several manual steps in a BP in the same way.