2. Get qualitative OCR result
Goal:
- Get a high-quality data set ready for tagging.
- Understand layouts' representation (i.e., number of documents for each layout) in order to collect a good training set and representative test set in future.
Input: CSV file with links to the original documents. All the documents are combined with no sorting by layouts.
Output: OCRed data set in which at least 90% of documents have acceptable quality.
Generate links to original documents
Click here for detailed step-by-step instruction.
Validate initial OCR quality
Before sending all documents to OCR Business Process (609 in this example), it’s necessary to make sure that current OCR settings are sufficient to ensure 90%+ quality of the data set after OCR.
To assess the quality of OCR, we prepare a sample set: randomly choose 20% of the data set (in this example, 122 documents) and OCR them.
Analyze results
When analysis of each record is completed, it’s necessary to calculate the percentage of documents with acceptable quality. In our case, out of 122 sample documents, only 10 have unacceptable quality after OCR and the remaining 112 (91.8% of the sample set) have no OCR problems, so overall quality of OCRed documents is even higher than the required minimum of 90%.
Refer to OCR results analysis section.
Although it's necessary to remove the 10 unacceptable documents with OCR issues from the data set, we can run the OCR business process for the whole data set without any additional OCR tuning.
When removing documents with unacceptable quality, pay attention to what type of template they belong to. If unacceptable quality items are mostly documents of one template, their exclusion may affect the representation of this template in the data set and additional documents should be requested. In this situation, the Delivery Manager should be notified.