Edit

Labeling logic definition

Goal of the step: Define correct tagging patterns (value shape, value boundaries, position in the document, context and acceptable/unacceptable cases due to structural changes after OCR) for each field in each layout.

Output of the step: Precise unambiguous instructions for every field in each batch.

In order to guarantee high quality of the data set and lower the number of tagging mistakes, it's necessary beforehand to define precise and unambiguous logic for each field in each batch so that the SME can clearly, without doubts, understand where and how to tag a field.

Steps

Identification of such logic requires thorough step-by-step analysis of documents from each batch:

1. Take about 20 samples from each batch.
2. Define for each field where and how to tag it. Take into account the factors described below.
3. Check that the defined rules correspond with tagging requirements.
4. Describe the logic with rules and screenshots in instructions.

Factors

None of the factors listed below should be ignored, as all of them together help identify the best place for each field to be tagged:

Number of field's occurrences

Field's occurrence means the number of times the same value of one field appears in the document. Usually the number of occurrences is considered for single-value fields. See the following example.

secretary_of_state field is presented in the document only once, so there will be no confusion in tagging this field and no specific instructions are required.

Address information (street, city, zip, etc.) is presented in two different blocks, so instruction should specify only one place to be used for this field.

Documents' structure

Structure means that after OCR, in the XML document, the information can be represented in the following ways:

• As a free text inside of one or several strings.
• As a table with different number of cells.
• With values divided into several parts.

Examples of different structures:

Value and surrounding context

For choosing specific occurrences which are going to be tagged, WorkFusion recommends considering the value itself and the surrounding context. Note: OCR quality is very important for choosing the specific occurrence of a field for tagging.

OCR quality

Both the value and surrounding context should be the most stable after OCR, this means:

• always recognized
• not corrupted
• no mistakes at all or only minor mistakes (which are systematic and can be fixed by applying a common rule)
• has the same structure

Content

Let us unambiguously identify the value:

• WorkFusion always means "WorkFusion".
• Workfusion always means "WorkFusion".

But WF may not always mean "WorkFusion".

• WF sometimes means "World Freight Group", sometimes "WorkFusion".
• CC sometimes means "Coca-Cola", sometimes means "Chemours Company".
• UBS can be interpreted as:
  • "UBS acting through its London branch"
  • "UBS Russia"
  • "UBS Group AG"
  • etc.

Content should be informative, uniform, unambiguous across all documents.

It is also better when there are some key words helping to identify the correct value, like:

• "number"
• "ID"
• "name"
• "client"
• etc.

Example of tagging logic development

We are not going to consider 20 documents now — let’s take just look at four to show the principle. Let's define specific occurrence for tagging business_id field for a batch (New Hampshire).

Document A

Original

OCRed

Document B

Original

OCRed

Document C

Original

OCRed

Document D

Original

OCRed

We have two occurrences for business_id.

Content factors have the same characteristics in both cases:

• Value is unambiguous in both cases, i.e. it always means specific business_id and it can not be interpreted in any other way.
• The content of surrounding context is good in both cases. For every occurrence, it goes after the words “Business ID” so it is good.

OCR quality factor.

To analyze it quickly, let’s create a table:

Document	1st occurrence OCR quality	2st occurrence OCR
A	+	(table)
B	+	(not a table)
C	+	- (not recognized)
D	+	- (context structure is different)

So, in this case it’s clear that it is better to choose the first occurrence, as it is more stable from the OCR quality point of view, while the other characteristics are the same.

As a result, the most convenient way to deliver this information is to provide a screenshot to the user, like this: