Skip to main content
Version: 10.2.9

Train SMEs

Goal of the step: Train Subject Matter Experts (SMEs) to provide a high level of labeling accuracy for collecting a quality dataset.

Output of the step:  SMEs learn how to label documents correctly and get qualified. The qualification process is mainly to prove that SMEs understand requirements correctly and can follow instructions well.

For SME training, you need the following:

Create gold data​

The gold data creation flow is as follows:

1. Take 15 documents of the same type and layout. Those can be 15 invoices from one supplier, for example.

  1. Define correct values for all fields and generate a CSV file with the links to the original and OCR'ed documents.

  2. Upload the file into the Manual Task you created to qualify workers. Note that gold data must be 100% correct from the business logic side.

  3. Label the gold data as described below:

    1. In the Manual Task to be used for qualifying workers, click Upload Data to upload the CSV file from Step 2.
    2. Go to the Run tab and select Run This Task.
    3. Find the task in Workspace, complete, and submit it.
    4. Generate Snapshot > download the CSV file > add to all column headers the gold_ prefix.
    5. Save and download the CSV file.
info

Open and edit CSV files in special editors, such as Ron's Editor or LibreOffice. The Excel editor breaks the structure of the snapshot.

Set up Qualification Business Process​

The qualification mechanism is no longer available for access control purposes and is replaced by a new user management approach. For the employee training, implement a special Qualification Business Process.

Train SMEs​

Firstly, provide basic information about Workspace:

  • How to log into Workspace
  • How to use Preview to look through the original document
  • How to label quickly using hotkeys
  • How to select values from lists
  • How to correct data values, and so on

Secondly, get workers to know the following common rules and principles of labeling:

  • Label fields consistently—in the exact places as specified in instructions.
  • Label all available fields, except for those corrupted by OCR.
  • Always label ALL products when using multi-product lines.
  • Never label an amount together with a currency sign.
  • Label fields completely. Never cut off the words from values, especially for company names, addresses, and so on.
  • Be careful with values in answer boxes. Select them from a drop-down list if there is one, or correct values as appropriate. Make sure not to submit incorrect answers.
  • Use reference data correctly. No N/A if you need to choose a value from a drop-down list.
  • Before submitting, check the position and data value of every label.

For more information about labeling requirements and collecting quality datasets, refer to the guide.

You can also download a sample presentation on SME training.