OCR

info

The article dwells on the obsolete approach of OCR usage in the scope of the ODF 2 framework.

We recommend using the OCR Bridge step, which is a more convenient solution in terms of performance and usability. For more details, refer to OCR usage example.

Take into consideration that the OCR-related services described on this page will be deprecated in the upcoming releases and moved to a separate module outside the ODF 2 core.

In ODF 2, there are two levels of API you can use to interact with the OCR service. The high-level API is a couple of Bot Tasks to submit documents and receive OCR results. It is the primary solution in most cases. When it is not enough, the low-level API is a service used to make calls directly to OCR.

High-level API

Submit documents for recognition

To use the OCR service, implement the SubmitDocumentToOcrTask interface.

SubmitDocumentToOcrTask

public interface SubmitDocumentToOcrTask extends TransactionProcessingTask {

    @Override
    default Class<? extends AbstractTransactionProcessingTaskRunner<?>> getRunnerClass() {
        return SubmitDocumentToOcrTaskRunner.class;
    }

    OcrInputDocument getDocument(S3Service s3Service, UUID transactionId);

    OcrConfiguration getOcrConfiguration();

    OcrService getOcrService();

}

To submit a document, the framework needs to know the following:

• A document to be processed
• The processing configuration
• Exact OcrService to be used

For that reason, the task interface has three methods to be correctly implemented.

In most cases, the document is located on the S3 server, and its URL is stored in the AI Agent-specific Data Model. Thus, the getDocument() method implementation must deal with getting the document URL out of the corresponding entity and then letting the task know how to read the document's content.

Since OcrInputDocument is a byte array Supplier, it can be bound to any source of byte data. Typical scenarios are as follows:

• Get the document from S3Service using the String URL:

  @Override
  public OcrInputDocument getDocument(S3Service s3Service, UUID transactionId) {
      String documentUrl = ourDocumentRepository.find(transactionId).getUrl();
      return () -> s3Service.getObjectByUrl(documentUrl);
  }

  The endpoint of the requested URL must match the one specified in preconfigured S3ConnectionProperties associated with the running Bot Task. The URL can point to any available bucket within the S3 server.

• Get the document from S3Service using the S3 bucket and key:

  @Override
  public OcrInputDocument getDocument(S3Service s3Service, UUID transactionId) {
      DocumentEntity entity = ourDocumentRepository.find(transactionId);
      String s3Bucket = entity.getS3Bucket();
      String s3Key = entity.getS3Key();
      return () -> s3Service.getBucket(s3Bucket).get(s3Key);
  }

• Get the document using a custom data source:

  @Override
  public OcrInputDocument getDocument(S3Service s3Service, UUID transactionId) {
      String documentId = ourDocumentRepository.find(transactionId).getDocumentId();
      return () -> customDataSource.getDocumentContent(documentId);
  }

The OCR configuration is a bunch of mostly self-explanatory flags stored in the com.workfusion.odf2.service.ocr.OcrConfiguration class. The corresponding task method must return an instance of the object configured for specific needs:

@Override
public OcrConfiguration getOcrConfiguration() {
  final OcrConfiguration configuration = new OcrConfiguration(OcrType.TOD);
  configuration.setLanguage("French");
  return configuration;
}

Inject a service and return it from the corresponding method:

// in constructor
@Inject
public SubmitToOcrExampleTask(@OcrSdk12 OcrService ocrService) {
  this.ocrService = ocrService;
}

// and later
@Override
public OcrService getOcrService() {
  return ocrService;
}

The resulting task looks like this

package com.workfusion.odf2.regression.task;

import java.util.UUID;
import javax.inject.Inject;
import org.slf4j.Logger;
import com.workfusion.odf2.compiler.BotTask;
import com.workfusion.odf2.core.cdi.Requires;
import com.workfusion.odf2.regression.bundle.RepositoryModule;
import com.workfusion.odf2.regression.bundle.repository.InputPictureRepository;
import com.workfusion.odf2.service.ControlTowerServicesModule;
import com.workfusion.odf2.service.ocr.OcrConfiguration;
import com.workfusion.odf2.service.ocr.OcrInputDocument;
import com.workfusion.odf2.service.ocr.OcrSdk12;
import com.workfusion.odf2.service.ocr.OcrService;
import com.workfusion.odf2.service.ocr.OcrType;
import com.workfusion.odf2.service.s3.S3Service;
import com.workfusion.odf2.transaction.task.ocr.SubmitDocumentToOcrTask;

@BotTask
@Requires({RepositoryModule.class, ControlTowerServicesModule.class})
public class SubmitPictureOcrTask implements SubmitDocumentToOcrTask {

    private final InputPictureRepository inputPictureRepository;
    private final OcrService ocrService;
    private final Logger logger;

    @Inject
    public SubmitPictureOcrTask(InputPictureRepository inputPictureRepository, @OcrSdk12 OcrService ocrService, Logger logger) {
        this.inputPictureRepository = inputPictureRepository;
        this.ocrService = ocrService;
        this.logger = logger;
    }

    @Override
    public OcrInputDocument getDocument(S3Service s3Service, UUID transactionId) {
        String documentUrl = inputPictureRepository.findByTransactionId(transactionId).get(0).getOriginalDocumentLink();
        logger.info("Document submitted for OCR: {}", documentUrl);
        return () -> s3Service.getObjectByUrl(documentUrl);
    }

    @Override
    public OcrConfiguration getOcrConfiguration() {
        return new OcrConfiguration(OcrType.STANDARD);
    }

    @Override
    public OcrService getOcrService() {
        return ocrService;
    }
}

Receive results

As soon as a document is submitted to OCR, some time passes before its processing. To get the resulting document, implement ReceiveOcrResultTask that deals with waiting for the processing under the hood. Write the code related to the received document and provide the framework with correct OcrService as in the SubmitPictureOcrTask implementation.

The resulting task looks like this

package com.workfusion.odf2.regression.task;

import java.util.UUID;
import javax.inject.Inject;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.slf4j.Logger;
import com.workfusion.odf2.compiler.BotTask;
import com.workfusion.odf2.core.cdi.Requires;
import com.workfusion.odf2.regression.bundle.RepositoryModule;
import com.workfusion.odf2.regression.bundle.model.InputPicture;
import com.workfusion.odf2.regression.bundle.repository.InputPictureRepository;
import com.workfusion.odf2.service.ControlTowerServicesModule;
import com.workfusion.odf2.service.ocr.OcrResult;
import com.workfusion.odf2.service.ocr.OcrSdk12;
import com.workfusion.odf2.service.ocr.OcrService;
import com.workfusion.odf2.transaction.TransactionResult;
import com.workfusion.odf2.transaction.task.ocr.ReceiveOcrResultTask;

@BotTask
@Requires({RepositoryModule.class, ControlTowerServicesModule.class})
public class RetrievePictureOcrResultTask implements ReceiveOcrResultTask {

    private final OcrService ocrService;
    private final InputPictureRepository inputPictureRepository;
    private final Logger logger;

    @Inject
    public RetrievePictureOcrResultTask(@OcrSdk12 OcrService ocrService, InputPictureRepository inputPictureRepository, Logger logger) {
        this.ocrService = ocrService;
        this.inputPictureRepository = inputPictureRepository;
        this.logger = logger;
    }

    @Override
    public OcrService getOcrService() {
        return ocrService;
    }

    @Override
    public void doWithResult(OcrResult ocrResult, UUID transactionId, TransactionResult result) {
        result.withColumn("meta_info_json", toJson(ocrResult.getOcrMetaInfo()));
        result.withColumn("document_xml_link", ocrResult.getDocumentXmlLink());
        result.withColumn("pages", String.valueOf(ocrResult.getPages()));
        
        InputPicture inputPicture = inputPictureRepository.findByTransactionId(transactionId).get(0);
        inputPicture.setOcrResultUrl(ocrResult.getDocumentXmlLink());
        inputPictureRepository.update(inputPicture);
        logger.info("Input picture {} OCR results are saved.", inputPicture.getName());
    }

    private static String toJson(Object obj) {
        try {
            return new ObjectMapper().writeValueAsString(obj);
        } catch (JsonProcessingException e) {
            throw new IllegalStateException(e);
        }
    }

}

Compose OCR in Business Process

Once you deploy your ODF 2 project into Control Tower which contains the above ODF Bot Tasks, you can drag-and-drop and connect them as follows:

Low-level API

If the functionality of out-of-the-box tasks is not enough, you can inject the com.workfusion.odf2.service.ocr.OcrService and com.workfusion.odf.ocr.client.OcrClient classes.

OcrService encapsulates the logic of interacting with OCR and S3 endpoints, storing input and output documents, OCR result parsing, and so on.

OcrClient is an HTTP client for the OCR endpoint. You can use it to implement your services of any desired complexity.

OCR cache

The ODF 2 framework provides the ability to cache OCR results. It helps to speed up a Business Process when dealing with repetitive documents as OCR itself is quite a time-consuming operation. Also, it helps to save your OCR license since retrieving documents from the cache does not require sending anything to the actual OCR server.

The OCR cache is available from the high-level API perspective and is enabled by default. To start using it, no additional actions are required. When you inject OcrService to a Bot Task, the OcrService implementation already comes with all the required functionality.

To illustrate how it works, let’s imagine you send a document to OCR for the first time. In this case, OcrService does the following:

• generates a cache key based on the document content and the specified OcrConfiguration
• sends the document to the OCR server
• retrieves OCR results and puts them into the uc_<uc_code>_ocr_cache_v<version> Data Store

When the same document with the same configuration is sent to OCR for the second time, OcrService does the following:

• generates a cache key based on the document content and the specified OcrConfiguration
• looks for the generated cache key inside the uc_<uc_code>_ocr_cache_v<version> Data Store and returns the associated result from the Data Store

note

All the binary files are stored on the S3 server. The uc_<uc_code>_ocr_cache_v<version> Data Store contains document links and meta information only.

If, for some reason, you switch the OCR cache off, use a special flag for this. Navigate to the uc_<uc_code>_config_v<version> Data Store and make sure the odf.ocr.cache.enabled option is set to false. Then, all upcoming requests from OcrService go directly to the OCR server without any caching logic.

To clean one or multiple cache entries, open uc_<uc_code>_ocr_cache_v<version> the Data Store and clean related records. It is a safe operation, and no data is lost as it contains document links and meta information.

If you require to operate cache entities programmatically, use the com.workfusion.odf2.service.ocr.OcrCache interface. Once it is injected into a Bot Task, it provides the following methods:

• boolean contains(String key) returns true if the cache contains an entry associated with the provided key, otherwise—false.
• Optional<OcrResult> get(String key) returns the OcrResult value associated with the provided key in the cache or Optional.empty() if there is no cached value for the key.
• void put(String key, OcrResult ocrResult) associates the OcrResult value with the key in the cache. If the cache previously contained a value associated with the key, the old value is replaced by a new one.
• void invalidate(String key) discards any cached value for the provided key.
• void invalidateAll() discards all entries in the cache.
• long size() returns the number of cache entries.