OCR analysis and tuning

The goal of this stage is to achieve acceptable OCR configuration and get understanding of final OCR quality.

Input: Original documents

Output:

• Effective configuration of OCR and all the documents pass through OCR process with acceptable final quality.
• Final report to Delivery Manager on OCR quality and all the risks identified (if any).

OCR overview

OCR (Optical Character Recognition) is the electronic conversion of images of typed, handwritten or printed text into machine-encoded text, whether from a scanned document, a photo of a document or from subtitle text superimposed on an image (for example, from a television broadcast).  

WorkFusion uses ABBYY OCR, which proposes an OOTB (out-of-the-box) solution: there is no necessity to write or maintain any custom code. However, it's a chargeable service and each page is paid for.

Today WorkFusion mostly works with ABBYY OCR, which is integrated into separate business processes.

The result of an OCR business process may look like this:

However, you may encounter the following result as well:

OCR quality factors

Such a discrepancy in the OCR output is justified because the quality of the final document is affected by factors both internal (OCR built-in parameters) and external (peculiarities of original documents):

Internal — OCR parameters

1. Export Format (usually XML)
2. Document language (language parameter must be supported by the license)
3. Brightness
4. Scanning resolution (DPI)
5. Contrast
6. Convert to black and white
7. Correct orientation
8. Allowed region type
9. Invert image (true, false)
10. Discard color image (true, false) etc.

External — documents' initial quality

See the examples of the most affecting factors.

Resolution (low, high)

Font size

Print quality (fax, newspaper)

Position on the page

If a page is too skewed, it may severely impact the quality of the OCR

Noise in the document

Noise like shadows, lines, dots left from previous copying/scanning.

Handwritten text

Logos, stamps, bar codes, watermarks

Crumpled sheets, stains

Recommendations

We recommend to use documents with high readability without visible defects and noise with initial quality of ~300 dpi. If these requirements are met, the documents after OCR look adequate and can be processed by ML.

The image above has good resolution, optimal font size and style, no noise and other defects, and thus will be well recognized.

note

It's necessary to understand that the result of the whole project directly depends on the OCR quality. Unacceptable OCR quality (i.e. missed and corrupted fields; corrupted structures, etc.) can be the most serious reason for bad ML model results from both training and testing aspects.

If the success criteria stated in the SOW is 90% accuracy and current OCR quality is about 90%, there is high possibility that final result will not meet the project requirements. OCR as one of the initial stages sets up the maximum which can be achieved on the project. All the rest of the stages may bring additional mistakes, which will only decrease the overall quality of ML results.

Running OCR business process

To launch the OCR business process, you need a file containing links to original documents. To get the links, follow the steps below.

Getting links

1. Open S3 browser.
2. Choose a bucket (or create a new one) and create a separate folder where original documents will be stored.
3. Click the Upload button, and choose Upload Folder(-s).
4. Select a folder on your computer with original documents and wait for all the records to be uploaded.
5. Set the permission to let all users read the files:
   1. Select all files.
   2. Click the Permissions tab.
   3. Select Read next to All Users.
   4. Click Apply changes.
6. Select all records in the folder (Ctrl+A), right-click the selected files, and then click Generate Web URL(s)..".
7. In the pop-up window, click Copy to clipboard, and verify that all the links were copied.
8. Open a new Excel file and paste the links.

Assessing OCR quality

Before sending all the available dataset to the OCR business process, it’s necessary to make sure that current OCR settings are sufficient to ensure the required quality of the dataset after OCR.

To assess the quality of OCR, it's recommended to prepare a sample set. Here, two variants are possible:

1. If a customer provided you with a dataset already sorted by some criteria (supplier, issuer, secretary of state, and so on), choose 10% from each group and OCR them.
2. If documents were provided without any sorting, randomly choose 10% of the dataset and OCR them.

Be prepared that after OCR, a document will differ from the original, especially documents with complicated structures (tables), font styles, stamp, logos, etc.

These are examples of input (original documents) and output (OCR documents):

Original document	OCR document	Template
Original doc. link	OCR doc. link	New Hampshire
Original doc. link	OCR doc. link	Massachusetts
Original doc. link	OCR doc. link	Georgia

Analysis of initial OCR results

To know what level of OCR quality we have, it's necessary to look through each document from the sample group (10 %) and fill in the report. The report should have links to the document (original and OCR) and an assessment of OCR quality: Acceptable/Unacceptable.

Acceptable: Each field and the context around it is recognized, mistakes can be corrected by post-processing and don't create any difficulties for extraction due to missing and/or incorrect and/or strange context or unrecognizable values.

Unacceptable: After OCR, at least one field can not be restored with post-processing, structure is corrupted, or the context around the field is not recognizable. More examples of unacceptable OCR quality can be found here. 

Count the percentage of Acceptable quality assessments. If it is higher than success criteria stated in the SOW, no tuning is needed. If not, additional analysis and adjustments should be done.

During analysis of OCR output, you may notice that not only does the OCR document differ from the original, but also documents which have the same original template may have different layouts after conversion.

tip

The group of documents after OCR which have a similar structure are called a layout. Before OCR, they're considered original documents.

Example of layouts

Below are two documents which belong to the same template, Georgia, and have the same fields and rows. However, after OCR they have different layouts.

Layout 1

Layout 2

info

These differences in layouts within one template should also be taken into account when preparing batches for labeling.

Look through original documents which have "Unacceptable" quality after OCR. Try to define problems which may cause low character recognition.

Tuning OCR

You may try different configurations of OCR parameters to get the best possible output quality. For example:

• Select the correct language
• De-skew images
  The program will automatically detect skewed pages and correct skew if necessary.
• Straighten text lines
  The program will automatically detect uneven text lines on images and straighten them without correcting trapezoid distortions.
• Correct image resolution
  The program will automatically determine the best resolution for images and will change the resolution of images when necessary.
• Fix inverted colors on image
  When appropriate, the program will invert the colors of an image so that dark text is printed on a light background.
• Convert to black and white
  The program will convert color images to black and white. This will greatly reduce the size of the resulting OCR project and speed up OCR. However, black and white conversion is not recommended for photos, magazine pages, and text in Chinese, Japanese or Korean.
• Remove color marks
  The program will detect and remove any color stamps and marks made in ink to facilitate the recognition of the text obscured by such marks. This option is designed for scanned documents with dark text printed on a white background. Do not select this option for digital photos and documents with color backgrounds.
• Adjust contrast
• Adjust brightness

ABBYY OCR suggestions

For various OCR problems ABBYY suggest the following solutions.

Characters are too thin and sketchy

• Lower the brightness to make the image darker.
• Use the grayscale scanning mode (brightness is adjusted automatically in this mode).

Characters are thick and stuck together

• Increase the brightness to make the image lighter.
• Use the grayscale scanning mode (brightness is adjusted automatically in this mode).

Scanning resolution

For best recognition results, vertical and horizontal resolutions must be the same.

Use the following resolution for scanning:

• 300 dpi for typical text (printed in fonts of size 10pt or larger).
• 400–600 dpi for texts (fonts size 9pt or smaller).

Setting the resolution too high (over 600 dpi) slows down OCR. Increasing the resolution beyond this point does not yield substantially improved OCR results. It should be adjusted.

Setting an extremely low resolution (less than 150 dpi) adversely affects OCR quality. It should be adjusted.

If the image has a non-standard resolution, it should be adjusted. Some faxes, for example, have a resolution of 204 by 96 dpi.

It's suggested to use color documents within OCR process. A color image, since it has different colors present, adds ability to remove RGB color channels prior to the processing, decreasing the document noise and providing a way to stabilize the image before applying grayscale conversion.

After OCR, tuning quality should be assessed again. Same as during the initial assessment, take another sample group of 10% of the dataset (different from the first one) and validate each record by filling in the report. Evaluate the volume of good-quality documents.

Try different combinations of OCR parameters to get the best possible result.

warning

If the initial documents' quality doesn't achieve a quality level sufficient for the model to meet SOW requirements despite any OCR configurations, this should be communicated to the customer as a potential risk. Such documents should be replaced with better ones or they should be processed separately with other (lower) success criteria.