Perform OCR health check
The OCR health check functionality allows monitoring the OCR Service state dynamically with certain intervals. You can retrieve results in the JSON format directly at the /api/v1/health-check path of the OCR Service API.
How it works
An OCR health check tests the following components, dependencies, or connectivity to them:
- Microsoft SQL Server
- RabbitMQ
- S3
- ocr-worker
- ocr-task
- ABBYY License
The OCR health check tries interacting with the components in a way close to that the OCR Service would use during its normal operation. When you request the /api/v1/health-check endpoint, it checks the components, collects and reports errors, warnings, and stats. When errors and warnings are absent, this indicates the fact that the OCR Service functions properly.
Configuration
Configure OCR health check
The OCR health check verifies individual components concurrently.
ocr.health.timeoutspecifies in seconds for how long the OCR health check waits for individual component validation. Effectively, this configuration property specifies how fast a health check responds. This also means that reports from all individual component checks are not completed before the timeout is represented in the report only partially.ocr.health.executor.pool.sizedefines how many component checks can be executed concurrently.ocr.health.executor.queue.capacitydefines how many component checks can be queued. These are advanced options. Generally, you will not need the former to be higher. The health check as of version 8.5 starts only five-component checks.
Default configuration values for application.properties of ocr-rest
ocr.health.timeout: 29
ocr.health.executor.pool.size: 10
ocr.health.executor.queue.capacity: 0
Configure Storage health check
S3 bucket is configured by ocr.bucket. It should have consistent values in ocr-rest and ocr-worker.
ocr.bucket: doc-upload
Configure message queue health check
When a message queue is tested, ocr-rest sends a request message to a request queue, while ocr-worker receives it, then sends a response message to a response queue, and finally, ocr-rest receives it. ocr-rest performs additional validation and retries if needed.
ocr.health.messaging.receive.timeoutspecifies in seconds for how long ocr-rest waits for a response message from ocr-worker.ocr.health.messaging.receive.retryCountspecifies how many times ocr-rest retries attempts to read a response message if a previous attempt was unsuccessful. ocr-rest uses an exponential backoff for the wait time before a consequent attempt to retry a receive operation.ocr.health.messaging.receive.maximumRetryIntervaldefines the maximum time.
If you need to redefine the RabbitMQ queue and exchange names used for the health check, you can use the corresponding properties mentioned below. Make sure the configurations are equal for ocr-rest.yml and ocr-worker.yml in application.properties.
If the rabbitmqQueue profile is active, the following parameters are applied to it:
ocr.health.messaging.requestQueueocr.health.messaging.requestExchangeocr.health.messaging.requestRoutingKeyocr.health.messaging.responseQueueocr.health.messaging.responseExchangeocr.health.messaging.responseRoutingKey
Default configuration values for application.properties of ocr-rest
ocr.health.messaging.receive.timeout: 1
ocr.health.messaging.receive.retryCount: 5
ocr.health.messaging.receive.maximumRetryInterval: 4
Default configuration values for application.properties of ocr-worker
ocr.health.messaging.executor.pool.size: 1
ocr.health.messaging.executor.queue.capacity: 0
Default configuration values for application.properties of ocr-rest and ocr-worker
# Health-Check : message queue with rabbitmqQueue
ocr.health.messaging.requestQueue: health_request_queue
ocr.health.messaging.requestExchange: health_request_exchange
ocr.health.messaging.requestRoutingKey: health_request_routing_key
ocr.health.messaging.responseQueue: health_response_queue
ocr.health.messaging.responseExchange: health_response_exchange
ocr.health.messaging.responseRoutingKey: health_response_routing_key
Configure worker health check
OCR workers periodically evaluate their health statuses and save the results to Microsoft SQL Server. When the /api/v1/health-check request is sent to ocr-rest by a client, ocr-rest collects the worker health checks from the database and provides a summary computed from those health checks.
The ocr.health.worker.cron property has a cron-like format and specifies the schedule of starting worker health checks. It has space-separated fields that define a repetition pattern for seconds, minutes, hours, days of the month, months, days of the week, respectively. The default schedule written below specifies to execute worker health check every 10 minutes: "0 0/10 * * * *". An extra health check is done right after ocr-worker starts. You can disable that by specifying ocr.health.worker.runOnStart=false .
If you have multiple ocr-worker instances that may have the same hostname, configure ocr.health.worker.name in ocr.yml (or the command line) for each ocr-worker instance to have a unique name among your ocr-worker instances.
During the worker health check, an ocr-task is executed (the actual text recognition process). Not to waste the license volume, actual recognition is not performed during a health check. However, ABBYY FREngine can be loaded during a task health check. This checks that ABBYY FREngine is installed properly and its license is available. Specify ocr.health.task.loadEngine=true to force loading of ABBYY FREngine during the task health check. ocr.health.task.timeout specifies a timeout in seconds for the ocr-task process during the task health check.
You can configure a worker health check to report warnings and errors depending on the expected number of healthy workers. If the number of workers that update their report timely (less than the ocr.health.worker.errorThreshold.reportTtl value in seconds) is less than the ocr.health.worker.errorThreshold.minHealthyCount value, an error is reported. The ocr.health.worker.warningThreshold.reportTtl and ocr.health.worker.warningThreshold.minHealthyCount properties have the same effect, with a warning being reported.
If ocr-worker is scaled down, you can tune the ocr.health.worker.cleanup.reportTtl property. The worker health reports older than the value are ignored entirely in the health check report.
Health check mechanism optimization
The ocr.health.worker.version parameter enables or disables a worker health check per a worker type in the following way:
ocr.health.worker.version=v1,v2—v1, v2 enabledocr.health.worker.version=v1—v1 enabledocr.health.worker.version=v2—v2 enabledocr.health.worker.version=—disabled
note
The default value is v1,v2, thus both versions are enabled for the health check.
To change the parameter, go to ZooKeeper using the /config/ocr-rest/ocr.health.worker.version path.
Default configuration values for application.properties of ocr-rest
ocr.health.worker.warningThreshold.reportTtl: 900
ocr.health.worker.warningThreshold.minHealthyCount: 1
ocr.health.worker.errorThreshold.reportTtl: 1800
ocr.health.worker.errorThreshold.minHealthyCount: 1
ocr.health.worker.cleanup.reportTtl: 3600
Default configuration values for application.properties of ocr-worker
ocr.health.worker.name: default_worker
ocr.health.worker.cron: 0 0/10 * * * *
ocr.health.worker.runOnStart: true
ocr.health.task.loadEngine: false
ocr.health.task.timeout: 60
ocr.health.task.debug: false
FREngine configuration is propagated automatically from ocr-rest to ocr-worker through Microsoft SQL Server. If ocr-worker starts earlier during the first launch, it can test itself before getting the configs from ocr-rest and report an error accordingly. The error disappears on the next check, once ocr-worker gets the configs.
Configure ABBYY license health check
During the license health check, the ABBYY license is loaded, and the remaining volume (the number of pages to be recognized included in the license) is checked.
If the remaining volume of pages is less or equal to zero, a warning is reported:
Your OCR license has expired. You have to purchase a new license in order to continue using OCR.
If the number of remaining pages is less than ocr.health.license.warningThreshold.minVolumeRemaining, a warning is reported:
Remaining number of pages to be processed under the current OCR license is below warning threshold: 500. Contact WorkFusion support team to obtain a new license.
Additionally, if the percentage of remaining pages is less than ocr.health.license.warningThreshold.minPercentageOfVolume, a warning is reported:
Your OCR license is about to expire. Less than 10% of pages left. You will need to purchase a new license in " + "order to continue using OCR.
Default configuration values for application.properties of ocr-rest
ocr.health.license.warningThreshold.minVolumeRemaining: 500
ocr.health.license.warningThreshold.minPercentageOfVolume=10
To change default values, create or update values for ocr-rest (/config/ocr-rest/) and ocr-worker (/config/ocr-worker/) in ZooKeeper, for example:
create /config/ocr-rest/ocr.health.license.warningThreshold.minVolumeRemaining 500
create /config/ocr-rest/ocr.health.license.warningThreshold.minPercentageOfVolume 10