AGS, AMS, AMMS services down
In AutoML, there are a lot of services and components:
In AutoML, there are a lot of services and components:
Possible causes of issues related to no visible progress in running Business Processes (BP) can be as follows:
To analyze task execution, follow the steps below:
[BEP] [Cluster] CPU
The article lists a number of hipotheses about the causes of the Worker performance degradation issues you are experiencing (if any). For each hypothesis, you'll find a description of symptoms, steps to reproduce, and root cause analysis (RCA) tips.
Code-related exceptions
The *vds-data/ NFS shared folder is used for sharing the resources within the BEP cluster.
These are issues due to failed processing by such services as Control Tower (CT), RabbitMQ, ZooKeeper (ZK), Task Dispatcher Service (TDS), Worker Management Service (WMS).
You can observe delays in task execution due to tasks waiting long for a free Worker in a cluster or delays in Worker starts due to cluster resources utilized by other task types, thus being insufficient.
The following symptoms occurring while running a model indicate that the problem might be not the model but the resources for running it:
You can understand BEP resources are lacking by the following symptom: you run OCR processing, and the results are not displayed for a long period (which is a really rare issue).
[BEP] [Messages] Input messages by client
[BEP] [Tasks by Worker type] Total tasks
[BEP] [Tasks Flow] Tasks Submitted
[BEP] [Tasks Time by Stage] By BP instance
[BEP] [Tasks Time by Stage] By BP Steps
[BEP] [Tasks by worker] Input Task
The document details recommendations of WorkFusion Support on troubleshooting issues related to the Work.AI infrastructure. Below is the sequence of the steps you need to take to locate and resolve an issue.
[BEP] [Worker] Absolute CPU Usage
BEP utilization percentage