Perform OCR health check
The OCR health check functionality monitors dynamically the OCR service state at specific intervals. You can retrieve results in the JSON format directly on the /api/v1/health-check path of OCR REST API.
Working principle
An OCR health check tests the following components, dependencies, or connectivity:
- Microsoft SQL Server
- RabbitMQ
- S3
- Ocr-worker
- Ocr-task
- ABBYY License
The OCR health check tries to interact with the components in a way close to that the OCR service would use during its normal operation. When you request the /api/v1/health-check endpoint, it checks the components, collects and reports errors, warnings, and stats. When you get no errors and warnings, this indicates that the OCR service functions properly.
Configuration
Configure OCR health check
The OCR health check verifies individual the above-mentioned components concurrently. To configure, follow the steps below:
ocr.health.timeoutspecifies how long in seconds the OCR health check waits for individual component validation. Effectively, the property determines how fast a health check responds. This also means that the reports from all individual component checks are not completed until the timeout is only partial.ocr.health.executor.pool.sizedefines how many component checks can be executed concurrently.ocr.health.executor.queue.capacityis an advanced option specifying how many component checks can be queued. The health check covers only the five components.
Default configuration values in application.properties for ocr-rest
ocr.health.timeout: 29
ocr.health.executor.pool.size: 10
ocr.health.executor.queue.capacity: 0
Configure storage health check
The S3 bucket is configured using the ocr.bucket parameter. It should have consistent values for ocr-rest and ocr-worker.
ocr.bucket: doc-upload
Configure message queue health check
When a message queue is tested, ocr-rest sends a request message to a request queue, ocr-worker receives it, and sends a response message to a response queue. Finally, ocr-rest receives the response, and ocr-rest validates it additionally, then retries if needed.
For the configuration, use the following parameters:
ocr.health.messaging.receive.timeoutspecifies for how long in seconds ocr-rest waits for a response message from ocr-worker.ocr.health.messaging.receive.retryCountspecifies how many times ocr-rest retries to read a response message if a previous attempt was unsuccessful. ocr-rest uses an exponential backoff for the wait time before another retry for the receive operation.ocr.health.messaging.receive.maximumRetryIntervaldefines the maximum interval between retries.
If you need to redefine the RabbitMQ queue and exchange names used for the health check, use the corresponding properties mentioned below. In application.properties, make sure the configurations are identical for ocr-rest.yml and ocr-worker.yml.
If the rabbitmqQueue profile is active, the following parameters are applied to it:
ocr.health.messaging.requestQueueocr.health.messaging.requestExchangeocr.health.messaging.requestRoutingKeyocr.health.messaging.responseQueueocr.health.messaging.responseExchangeocr.health.messaging.responseRoutingKey
Default configuration values in application.properties for ocr-rest
ocr.health.messaging.receive.timeout: 1
ocr.health.messaging.receive.retryCount: 5
ocr.health.messaging.receive.maximumRetryInterval: 4
Default configuration values in application.properties for ocr-worker
ocr.health.messaging.executor.pool.size: 1
ocr.health.messaging.executor.queue.capacity: 0
Default configuration values in application.properties for ocr-rest and ocr-worker
# Health-Check : message queue with rabbitmqQueue
ocr.health.messaging.requestQueue: health_request_queue
ocr.health.messaging.requestExchange: health_request_exchange
ocr.health.messaging.requestRoutingKey: health_request_routing_key
ocr.health.messaging.responseQueue: health_response_queue
ocr.health.messaging.responseExchange: health_response_exchange
ocr.health.messaging.responseRoutingKey: health_response_routing_key
Configure Worker health check
OCR Workers evaluate their health statuses periodically and save the results to Microsoft SQL Server. When a client sends the /api/v1/health-check request to ocr-rest, ocr-rest collects the Worker health check data from the database and provides a summary computed based on those.
The ocr.health.worker.cron property has a cron-like format and specifies the schedule for starting Worker health checks. It has space-separated fields that define a repetition pattern for seconds, minutes, hours, days of the month, months, days of the week, respectively.
The default "0 0/10 * * * *" schedule causes a Worker health check to be executed every 10 minutes. An extra health check is run right after ocr-worker starts. You can disable it by specifying ocr.health.worker.runOnStart=false .
If you have multiple ocr-worker instances that can have the same hostname, configure ocr.health.worker.name in ocr.yml (or the command line) for each instance to have a unique name.
During a Worker health check, ocr-task is executed (the actual text recognition process). Not to waste the license volume, the health check involves no actual recognition. However, ABBYY FREngine can be loaded during a task health check. This is essential to ensure that ABBYY FREngine is installed properly and its license is available. Specify ocr.health.task.loadEngine=true to force loading of ABBYY FREngine during the task health check. ocr.health.task.timeout specifies a timeout in seconds for the ocr-task process during the task health check.
You can configure a Worker health check to report warnings and errors, depending on the expected number of healthy Workers. If the number of Workers that update their report timely (less than the ocr.health.worker.errorThreshold.reportTtl value in seconds) is fewer than the ocr.health.worker.errorThreshold.minHealthyCount value, an error is reported. The ocr.health.worker.warningThreshold.reportTtl and ocr.health.worker.warningThreshold.minHealthyCount properties have the same effect, with a warning produced.
If ocr-worker is scaled down, you can tune the ocr.health.worker.cleanup.reportTtl property. In the health check report, any Worker health data older than the value are ignored.
Health check mechanism optimization
The ocr.health.worker.version parameter enables or disables a Worker health check per Worker type in the following way:
ocr.health.worker.version=v1,v2—v1, v2 enabledocr.health.worker.version=v1—v1 enabledocr.health.worker.version=v2—v2 enabledocr.health.worker.version=—disabled
note
By default, the ocr.health.worker.version value is empty, thus both versions are disabled for the health check.
To change the parameter, go to ZooKeeper and use the /config/ocr-rest/ocr.health.worker.version path.
Default configuration values in application.properties for ocr-rest
ocr.health.worker.warningThreshold.reportTtl: 900
ocr.health.worker.warningThreshold.minHealthyCount: 1
ocr.health.worker.errorThreshold.reportTtl: 1800
ocr.health.worker.errorThreshold.minHealthyCount: 1
ocr.health.worker.cleanup.reportTtl: 3600
Default configuration values in application.properties for ocr-worker
ocr.health.worker.name: default_worker
ocr.health.worker.cron: 0 0/10 * * * *
ocr.health.worker.runOnStart: true
ocr.health.task.loadEngine: false
ocr.health.task.timeout: 60
ocr.health.task.debug: false
The FREngine configuration is propagated automatically from ocr-rest to ocr-worker via Microsoft SQL Server. If during the first launch, ocr-worker starts earlier, it can test itself before getting the configs from ocr-rest and report an error accordingly. The error disappears during the next check, once ocr-worker gets the configs.
Configure ABBYY license health check
During the license health check, the ABBYY license is loaded, and the remaining volume (the number of pages to be recognized included in the license) is checked.
If the remaining volume of pages is less or equal to zero, the following warning is returned:
Your OCR license has expired. You have to purchase a new license in order to continue using OCR.
If the number of remaining pages is less than ocr.health.license.warningThreshold.minVolumeRemaining, the following warning is returned:
Remaining number of pages to be processed under the current OCR license is below warning threshold: 500. Contact WorkFusion support team to obtain a new license.
Additionally, if the percentage of remaining pages is less than ocr.health.license.warningThreshold.minPercentageOfVolume, a warning is returned:
Your OCR license is about to expire. Less than 10% of pages left. You will need to purchase a new license in " + "order to continue using OCR.
Default configuration values in application.properties for ocr-rest
ocr.health.license.warningThreshold.minVolumeRemaining: 500
ocr.health.license.warningThreshold.minPercentageOfVolume=10
To change default values, create or update values for ocr-rest (/config/ocr-rest/) and ocr-worker (/config/ocr-worker/) in ZooKeeper, for example:
create /config/ocr-rest/ocr.health.license.warningThreshold.minVolumeRemaining 500
create /config/ocr-rest/ocr.health.license.warningThreshold.minPercentageOfVolume 10