Create schemaless Business Process in 20 minutes

After reading this guide, you'll know how to prepare and run a Business Process with a trained model and a Manual Task.

Prerequisites

• You have an installed Work.AI instance.
• Control Tower is running.
• You have obtained a Business Process bundle from ITOps or downloaded it and imported into Control Tower.

Prepare source documents

To process the source documents, you must first upload them to the MinIO object storage.

To upload the documents, do the following:

1. Log in to the MinIO Console. To access it, request the URL, login, and password from your System Administrators.

   

2. Go to Buckets, find the doc-upload bucket, and click Browse next to it.

   tip

   Each bucket is a separate folder that stores data.

3. Inside the bucket, create a new folder, go to it, and click Upload to add your documents.

   tip

   Alternatively, you can drag and drop the documents directly to the open folder to upload them.

   

You can now close the MinIO Console and go back to Control Tower.

To pass the uploaded files' links to a Business Process, do the following:

1. Create a CSV file and populate it manually with the links to the documents added to the MinIO bucket earlier. Download a sample file to see how it should look.

2. In Control Tower, go to Business Processes, select your Process, and go to the Data tab.

3. Click Upload and add the CSV file with the links you have just created.

   

Configure Business Process

In this example, you will be running a BP consisting of the following steps:

• OCR
• OCR—Model adapter
• Extraction or Classification Model (AutoML)
• Model—Manual Task adapter
• Manual Task

To adjust the settings of your Business Process steps, in Business Process, go to the Workflow tab, and double-click each step. The configuration panel appears to the right of the workflow.

• OCR or OCR Bridge

  Here, you can leave the settings as they are.

  

• OCR-to-AutoML adapter

  Adapters communicate input and output among OCR and a model or the model and a Manual Task. They are needed as the outputs of each step are slightly different, so they have to be aligned to be processed in the next step.

  Here, you can also leave the settings as they are.

  

• AutoML

  Here, you configure a model responsible for document processing and previously trained in the AutoML UI.

  

  On the configuration panel, you can change the following settings:

  • AutoML-Model ID to process

    To get this value, in Control Tower, go to Models, find a required model you want to integrate into your Business Process and copy its ID.

    

  • Document for processing by model

    Specify the document.

  For the AutoML step, you can use a model developed in your other environments. For example, to apply a model from staging to production, do the following:

  1. In Control Tower in a staging environment, go to Models, find a model you want to export, open the Actions menu, and then select Export model to save it.

     

  2. Go to Control Tower in a production environment, open Models > the Plus sign > Import, and import the required model.

     

  3. Add the imported model to the Business Process.

• AutoML-to-Manual Task adapter

  The adapter retrieves output from the AutoML engine and converts it into two types of parameters. These steps are universal regardless of the data or model type.

  • meta_info_json. This data is passed to new Manual Tasks.
  • message_tagged. The parameter is required for legacy Manual Tasks, for example, those created in the legacy Manual Task designer, to make them compatible with the new flow.

  

  Here, you can leave the settings as they are and don't change the step when the Manual Task is changed.

• Manual Task

  Generally, a Manual Task is created in the Manual Task Designer and does not require customization or adding extra JavaScript code. Its configuration window usually reflects the data pulled from the model.

  note

  Pay extra attention when aligning your Manual Tasks in BP and MT Designer. You must label the fields manually. Note that the set of fields must be the same. You don't have to cover all the fields, but you can add only as many fields as you have from the model.

  

Besides these steps, you can also add others. For that, find a required step in the right panel on the Workflow tab. After adding a step, you can rename it to clean up your workflow. For example, change OCR Bridge to OCR.

Run Business Process

Once the configuration is complete, you are ready to run the Business Process:

1. Go to the Run tab, and click Run This Process.

   

2. On the Results tab, track the BP operation.

3. Click the step with the WS icon.

   

   The step statistics opens.

4. Click the WorkSpace link to go to the available assignments in the WorkSpace app.

   

5. See the assignment in WorkSpace:

   

Q&A

Can I use AutoML UI to change ML model?

A machine learning (ML) model is an immutable artifact that is locked for changes. To fine-tune a model, you need a new artifact with a different unique ID.

However, you can change the pipeline (hyper model).

By default, the WorkFusion platform contains several pre-uploaded pipelines, for example, Information Extraction (IE), designed to train IE models. These pipelines have basic pre- and post-processing capabilities and feature extraction. When you train a model in the AutoML UI, some of those capabilities for the post-processing are available through the user interface.

For example, when adding complex processors or feature extractors in more advanced cases, you have to customize them with JavaScript code. After doing so, you upload the pipeline to the platform to make it visible in the Pipelines menu. The general process remains the same: upload a dataset > have a custom pipeline > train a model > view the results.

What is Modify Model for?

Imagine that you want to create a model with custom post-processors. Instead of setting the model's configuration anew, select an existing model and click Modify Model to create a new one. All the steps in it are taken from the previous iteration. For details, see the Manage models guide.

What is best practice to train model in AutoML UI?

For the ML model training, it is recommended to get as many different vendor formats as possible. The model focuses on the vendor-provided document layouts or how the data is displayed in documents. So for your goals, it is best to train a model on a maximum possible number of unique layouts or page variations. For example, for training, you can use documents containing but not limited to:

• Tables with borders
• Tables without borders
• Tables on top and bottom
• Two, three, and more tables
• Different header variations
• Other elements

The general recommendation is to have at least ten different vendors. If you have fewer, most probably, your model won't work. Having 20-25 vendors is a reasonable compromise, while 100 of them is the most effective solution.

Most typical reason for ML model failure

The most common errors in ML models taken into production are caused by data drift. It happens when you train a model on one data type, but in other environments, it faces slightly different data. This difference leads to incorrect predictions. So, make sure that vendors' layout is similar in all environments.

Do I need to have various filetypes for successful model training?

For ML model training, file types do not matter. You can feed any type (for example, PDF, email, and so on) to the model.

Additional reading

• Post-Processing

TROUBLESHOOTING

In case a Business Process becomes non-operational or its performance degrades, look for troubleshooting tips in the following support guides:

• Investigate Business Process performance degradation

• Investigate non-operation Business Process

When you have trouble creating a BP, read the following guide:

• Impossible to create Business Process