Edit

Formats of input documents

Formats

The format of the input file to be used for model training is important both from the perspective of pre-processing, for example, digitizing it in case of PDF, and from the perspective of the training success. Let's consider some possible input formats and how to deal with them.

HTML

• Action: None.
• Post-action: Can be directly processed with AutoML.

Image

• Action: Send to OCR.
• Post-action: After OCR of an image or PDF, we receive output in two formats: XML and HTML. For ML, it is important to use XML as it contains more information about the original document. It is also advisable to use ABBYY OCR—its engine is used by default in WorkFusion—where possible because it produces XML that is most useful for WorkFusion ML models, for example, contains tags that are directly used by some of the ML features.

PDF

• Action: Digital PDF can be processed with Java code. PDF doesn't have a fixed format and new documents require an additional effort to process. Document formatting is lost in most of cases. Send to OCR.
• Post-action: Same as for images.

Excel

• Action: Rule-based approach using Java, and convert into HTML before processing with AutoML.
• Post-action: Excel input is complex. Excel files can be huge in size. After conversion to HTML, it might be an issue to display them in a Manual Task in WorkSpace. The right approach in such a case is to split Excel files by 50–100 rows before displaying these parts in a Manual Task and then combine the output back to what is required.

Word

• Action:
  • Get plain text using Java.
  • Convert into HTML using Java.
  • Convert into PDF or Image and send to OCR.
• Post-action: None.

Email

• Action: Extract an email body as text.
• Post-action: Work with the email body like with a plain text.

Plain text

• Action: None.
• Post-action: Use directly with AutoML.

ML vs rule-based approach

HTML

General approach: Rule-based.

For HTML files, we also can go with both approaches. Mind that if a customer has a well-structured HTML format, probably the best solution is to use XPath to extract data from documents.

Image

General approach: ML.

This document type usually comes as scans. For images, use the following workflow:

1. Make sure an image has sufficient resolution (at least 300 dpi) and convert it if needed using the ImageMagic library.
2. OCR step.
3. MT or ML step to extract information from the document.

PDF

General approach: ML.

PDF documents can be categorized into three different types, depending on the way a file originated. How it was originally created also defines whether the content of the PDF (text, images, tables) can be accessed or whether it is “locked” in an image of the page.

• “True” or Digitally Created PDFs: Digitally created PDFs, also known as “true“ PDFs, are created using software, such as Microsoft Word, Excel, or via the “print” function within a software application (virtual printer). They consist of a text and images. Both the characters in the text and the meta-information have an electronic character designation. With ABBYY FineReader 14, you can easily search through these PDFs and select, edit or delete text similar to how you would do that in other editable formats, such as Microsoft Word. Images in digitally created documents can be resized, moved, or deleted.
• Image-only or Scanned PDFs: When scanning hard copy documents on MFPs and office scanners, or when converting a camera image, a JPEG file, a TIFF file, or a screenshot into a PDF, the content is “locked” in a snapshot-like image. Such image-only PDF documents contain just the scanned or photographed images of pages, without an underlying text layer. Consequently, image-only PDF files are not searchable, and their text usually cannot be modified or marked up. An image-only PDF can be made searchable by applying OCR, with which a text layer is added, normally under the page image.
• Searchable PDFs
  Searchable PDFs usually result from applying OCR to scanned PDFs or other image-based documents. During the text recognition process, characters and the document structure are analyzed and “read." A text layer is added to the image layer, usually placed underneath. Such PDF files are almost indistinguishable from the original documents and are fully searchable. Text in searchable PDF documents can be selected, copied, and marked up. The PDF processing workflow depends on the PDF type. In case of searchable or "true" PDF, we can get the content of the files using the PDFBox Apache library. "Image-only" PDFs should go through an OCR step at first (the same workflow as for images).

Excel

General approach: Rule-based.

Both approaches can be applied for this document type, but generally, it's much easier to implement the rules-based one because Excel is a structured document.

The ML approach can be used if the customer has many different templates of Excel documents. Please note there may be some issues while viewing and extracting data in the Manual Task step for a document with more than 50 rows. In addition to this, keep in mind that it's best to convert Excel into HTML before sending the document to a Manual Task.

Plain text

General approach: ML.

The ML approach is preferred for this format, but the rules-based one also can be applied, for example, if 100% sure invoice number is the first word in the document. Note that plain text is the worst case for the ML approach because this format doesn't have any additional information like HTML tags.

Other formats

You may encounter other types of documents. Review the structure of the documents to make the right decision on the use of an approach.