OCR usage example · WorkFusion Documentation

The article is an example of OCR usage in the scope of the ODF 2 framework. The Bot Tasks used in the example are a part of Processing Business Process from the example project.

Before getting into the details of the OCR tasks, let's make a quick description of the Business Process:

The Business Process starts with a Processing Monitor that picks Transactions finalized by the Intake Business Process.
SplitByEmailAttachmentsTask creates an Invoice object for each Attachment in a given Email.
SubmitInvoiceToOcrTask and RetrieveInvoiceOcrResultTask are responsible for character recognition of PDF-related invoices. These two tasks are the main topic of this article.
InvoiceProcessorTask showcases the business entity processing logic.
At the end of the Business Process, there is a default finalization task.

note

The current OCR example is concentrated on using the asynchronous approach and does not touch on such topics as Low-level API or OCR cache.

Refer to OCR API for more information about OCR usage.

Submit document to OCR

To submit a document to OCR, use the com.workfusion.odf2.transaction.task.ocr.SubmitDocumentToOcrTask interface and the related SubmitDocumentToOcrTaskRunner from the ODF 2 core. The runner provides all the required logic of sending documents to OCR. Still, some methods shall be implemented at our end:

OcrService getOcrService() specifies the exact OcrService to be used. There are two versions of OCR SDK you can use in production: @OcrSdk12 and @OcrSdk11. @OcrSdk12 is used in the example.
OcrConfiguration getOcrConfiguration() defines the processing configuration. OcrConfiguration is described in detail in the upcoming section. For the example, choose the default configuration and OcrType.TOD type.
OcrInputDocument getDocument(S3Service s3Service, UUID transactionId) provides a document to be processed. In the example, you get a document content from S3 using S3Service that is available as a method argument.

@BotTask
@Requires({RepositoryModule.class, ControlTowerServicesModule.class})
public class SubmitInvoiceToOcrTask implements SubmitDocumentToOcrTask {

    private final CurrentTransaction transaction;
    private final InvoiceRepository invoiceRepository;
    private final OcrService ocrService;
    private final Logger logger;

    private Invoice invoice;

    @Inject
    public SubmitInvoiceToOcrTask(CurrentTransaction transaction, InvoiceRepository invoiceRepository,
            @OcrSdk12 OcrService ocrService, Logger logger) {
        this.transaction = transaction;
        this.invoiceRepository = invoiceRepository;
        this.ocrService = ocrService;
        this.logger = logger;
    }

    @Override
    public void afterConstruction() {
        invoice = transaction.getId()
                .map(invoiceRepository::findByTransactionId)
                .orElse(Collections.emptyList())
                .stream().findFirst()
                .orElse(null);
    }

    @Override
    public boolean shouldRun() {
        return invoice != null
                && "PDF".equals(invoice.getType())
                && StringUtils.isNotEmpty(invoice.getOriginalDocumentUrl());
    }

    @Override
    public OcrInputDocument getDocument(S3Service s3Service, UUID transactionId) {
        logger.info("Sending invoice '{}' to OCR processing", invoice.getUuid());
        return () -> s3Service.getObjectByUrl(invoice.getOriginalDocumentUrl());
    }

    @Override
    public OcrConfiguration getOcrConfiguration() {
        return new OcrConfiguration(OcrType.TOD);
    }

    @Override
    public OcrService getOcrService() {
        return ocrService;
    }

}

You may notice that the task contains two additional methods: afterConstruction and shouldRun. They are a part of the execution lifecycle methods and help to define whether the current Transaction has to be processed or not.

In our case, the Invoice and the related document are sent to OCR processing only when:

The current Transaction is present and InvoiceRepository contains the associated Invoice entity.
The Invoice entity has a PDF type.
The Invoice entity has a document link.

Here is what happens under the hood when SubmitInvoiceToOcrTask is executed:

// (1)
OcrConfiguration ocrConfiguration = task.getOcrConfiguration();
OcrService ocrService = task.getOcrService();
OcrInputDocument document = task.getDocument(s3Service, currentTransaction.getRequiredId());

// (2)
OcrTaskData taskData = ocrService.sendDocumentToOcr(ocrConfiguration, document);

// (3)
result.withColumn(TaskVariable.OCR_TASK_ID.toString(), taskData.getTaskId());

The runner takes OcrService, OcrConfiguration, and OcrInputDocument from the task.
The configuration and the document are delegated to OcrService.
The runner exports the OCR task ID for the OCR result retrieving on the next bot step.

After SubmitInvoiceToOcrTask is completed, and a document is sent for OCR processing, the Transaction goes to the next step.

Receive OCR result

To get a resulting document, use the com.workfusion.odf2.transaction.task.ocr.ReceiveOcrResultTask interface and the related ReceiveOcrResultTaskRunner from the ODF 2 core. The runner deals with waiting for the processing under the hood. A receiver Bot Task has to implement the following methods:

OcrService getOcrService() specifies the exact OcrService to be used. Must be the same as in SubmitInvoiceToOcrTask.
void doWithResult(OcrResult ocrResult, UUID transactionId, TransactionResult result) is a callback method executed when an OCR result is ready.

In the example, when the OCR processing is completed, retrieve an Invoice entity associated with the current Transaction from InvoiceRepository and update the related field with the OCR result. The upcoming Bot Tasks might use this information to complete the processing.

@BotTask
@Requires({RepositoryModule.class, ControlTowerServicesModule.class})
public class RetrieveInvoiceOcrResultTask implements ReceiveOcrResultTask {

    private final OcrService ocrService;
    private final TaskInput taskInput;
    private final InvoiceRepository invoiceRepository;
    private final Logger logger;

    @Inject
    public RetrieveInvoiceOcrResultTask(@OcrSdk12 OcrService ocrService, TaskInput taskInput, InvoiceRepository invoiceRepository, Logger logger) {
        this.ocrService = ocrService;
        this.taskInput = taskInput;
        this.invoiceRepository = invoiceRepository;
        this.logger = logger;
    }

    @Override
    public boolean shouldRun() {
        return taskInput.getVariable(TaskVariable.OCR_TASK_ID).isPresent();
    }

    @Override
    public OcrService getOcrService() {
        return ocrService;
    }

    @Override
    public void doWithResult(OcrResult ocrResult, UUID transactionId, TransactionResult result) {
        Invoice invoice = invoiceRepository.findFirstByTransactionId(transactionId);
        invoice.setOcrResultUrl(ocrResult.getDocumentXmlLink());
        invoiceRepository.update(invoice);

        logger.info("OCR result received for invoice '{}'", invoice.getUuid());
    }

}

Also, override the shouldRun method to make sure that the task is run only when a task input contains a special TaskVariable.OCR_TASK_ID variable. This variable is set by SubmitInvoiceToOcrTask in case of a successful submission.

Here is what happens under the hood when RetrieveInvoiceOcrResultTask is executed:

// (1)
String taskId = taskInput.getRequiredVariable(TaskVariable.OCR_TASK_ID);
OcrService ocrService = task.getOcrService();
if (ocrService.isOcrTaskCompleted(taskId)) {
    // (2)
    OcrResult ocrResult = ocrService.getOcrResult(taskId);
    task.doWithResult(ocrResult, currentTransaction.getRequiredId(), result);
    result.withoutColumn(TaskVariable.OCR_TASK_ID.toString());

} else {
    // (3)
    task.cancelAndRetryAfter(task.getRetryInterval());
}

The runner takes OcrService from the task and checks whether the OCR processing is completed based on the TaskVariable.OCR_TASK_ID variable.
If the processing is completed successfully, the runner retrieves the OCR result and executes the doWithResult callback method. Then, TaskVariable.OCR_TASK_ID is removed from the export variables.
If the processing is not completed, the task is retried using the ODF 2 retry flow in one minute. To change the retry interval, override the getRetryInterval method.
If the processing fails, OcrService throws OdfFrameworkException with the relevant message. Use the built-in Exception handling mechanism to handle such a case.

note

The ODF 2 retry flow means that a task is returned to Control Tower and re-submitted to BEP at a specified interval.

The approach releases a worker and lets it process other tasks while an OCR result is not ready.

Configure OCR

com.workfusion.odf2.service.ocr.OcrConfiguration is a DTO object that contains all the related OCR settings for the OCR submission. Follow the link to get more details about OCR REST API.

When creating an OcrConfiguration object, specify its type that defines default values for some properties:

OcrType.TOD: choose when Tagging Over Document is required.
OcrType.STANDARD: choose for all other cases.

See the description and default values of each property in the order of appearance in OcrConfiguration:

Value	Default	Description
`exportFormat`	`OcrConfiguration.DEFAULT_EXPORT_FORMAT`	OCR API export format. You can use a maximum of three types at once. Supported types: `OcrExportType.HTML`: HTML page. `OcrExportType.PDF_SEARCHABLE`: text that can be searched in such a file. `OcrExportType.XML`: file contains characters or words and their location in the original document (coordinates or frames). `OcrExportType.XML_FOR_CORRECT_IMAGE`: the same as XML, except the location is taken from a processed or adjusted document. `OcrExportType.TXT`: plain text (default).
`correctSkew`	`true`	If `true`, the page skew is detected and automatically corrected.
`xmlWriteRecognitionVariants`	`false`	If `true`, makes XML and xmlForCorrectedImage formats contain all variants of character or word OCR considers as a possible recognition.
`lowResolutionMode`	`false`	Improves recognition of images with low resolution, for example, faxes. If `true`, adapts to a low resolution image.
`profile`	For `OcrType.TOD`: `documentConversion` For other cases: not set	Allows fine-tuning the Engine. Supported profiles are as follows: `documentConversion`: to convert documents into editable formats, optimized for accuracy. `textExtraction`: to extract text from documents, optimized for accuracy. `barcodeRecognition`: to extract barcodes, optimized for accuracy.
`language`	Not set	Specifies predefined language, for example, `English`. You can also define multiple languages and use comma as a separator: `English,German,Polish`.
`engine`	Not set	OCR engine to be used. Currently, can be only `ABBYY`.
`correctOrientation`	`true`	Specifies whether the image orientation should be automatically detected and corrected. If `true`, the page orientation is detected, and if it differs from normal, it is automatically rotated.
`customRegions`	JSON variable, not set	Defines the area of a special type. Follow the OCR REST API to get supported types information and JSON examples.
`skipPreprocessing`	`false`	Disables all pre-processing steps for a document. The following parameters are disabled if `skipPreprocessing=true`: `convertTo` `changeDPI` `useAutoDetectedDPIFromRange` `removeNoiseModels` `removeGarbageSize` `removeColorObjectsType` `removeColorObjects` `correctSkew` `invertImage` `discardColorImage` `enhanceLocalContrast` `correctOrientation` `lowResolutionMode` `useWordsFromDictionaryOnly`
`useOnlyCustomRegions`	`false`	Skips the original analyzing stage to process custom regions only and extracts information from custom regions only.
`useWordsFromDictionaryOnly`	`false`	Specifies if dictionary words are allowed during recognition in this base language only. If `true`, a word that is not found in the dictionary of the base language can appear in the recognized text only if ABBYY FineReader Engine finds no dictionary variants.
`dictionary`	Not set	Word or a combination of characters that can be used to improve OCR recognition. The set of words extends but not limits the default dictionary.
`alphabetExtension`	Not set	String with special symbols to extend the already defined alphabet, for example, `abc123`.
`pattern`	Not set	OCR pattern to be used for recognition of special symbols.
`allowedRegionTypes`	`BT_Table,BT_Text,BT_Separator,` `BT_SeparatorGroup,BT_AutoAnalysis`	Sets the allowed region types.
`customAlphabet`	Not set	String with special symbols to extend the already defined alphabet, for example, `abc123`.
`skipTextLayerExtraction`	Not set	Text layer of the source PDF file is not used, the image layer is recognized by ABBYY FineReader Engine.
`removeNoiseModels`	Not set	Removes noise on the image. A valid value refers to comma-separated values: `CorrelatedNoise`, `WhiteNoise`). The method can be used for color and 8-bit gray images only.
`removeGarbageSize`	Not set	Removes garbage (excess dots smaller than a certain size) from the image. Valid value > 0 and -1 for automatically detected garbage size.
`useDefaultPattern`	`false`	If `true`, it requires applying the default pattern from the OCR application bundle.
`discardColorImage`	`false`	If you work with black-and-white images or the color of images is not important, set the `discardColorImage` to `true`.
`enhanceLocalContrast`	`false`	Specifies whether the local contrast of the image should be increased. Such preprocessing may increase the quality of recognition.
`priority`	`0`	Integer number that defines priority in the image processing queue.
`changeDPI`	Not set	Contains a new value of DPI, changeDPI available values from 50 to 3200, for example, `changeDPI=300`.
`invertImage`	`false`	Inverts image colors.
`removeColorObjects`	Not set	Removes color objects from the image, color values: Blue, Green, Red, Yellow. The parameter applies color filtering to the color image plane of the Document. Therefore, it’s incompatible with the `discardColorImage` parameter.
`removeColorObjectsType`	Not set	Specifies the type of objects to be removed. Supported types: `Full` `Background` `Stamp`
`storePreprocessedDoc`	For `OcrType.TOD`: `true` For other cases: `false`	Specifies whether preprocessed documents should be saved into the data storage.
`timeout`	60 seconds	Sets up a timeout in seconds between task execution.