AGS, AMS, AMMS services down
In AutoML, there are a lot of services and components:
In AutoML, there are a lot of services and components:
The steps below describe the circumstances how the issue occurs:
To increase the extract timeout, change the Nginx configuration files listed below.
Possible root causes for AutoML-related Business Process performance issues include changes in input data or utilization of cluster resources. If you suspect an issue at the AutoML step, act as follows:
You get org.springframework.beans.factory.BeanCreationException. This means an error occurred while a bean was being created with the zookeeperProperties name defined in org.springframework.cloud.zookeeper.ZookeeperAutoConfiguration. The bean initialization failed. There's a nested exception in java.lang.
BP error with no response for 280 ms
An Eval job finished with the FAILED status because the Marathon job was killed or there is no process or process.error file. To investigate, act as follows:
The troubleshooting instruction is only valid for Work.AI versions up to 10.3.2. In 10.3.2, Python 3.7 was deprecated.
Case 1: model failed to extract data
The *vds-data/ NFS shared folder is used for sharing the resources within the BEP cluster.
The guide is valid for versions up to v10.2.8. From 10.2.9, the Cognitive Bot Training Business Process is obsolete.
The following symptoms occurring while running a model indicate that the problem might be not the model but the resources for running it:
Machine learning (ML) models are supposed to work the same way in any environment. To figure out what is wrong when a model gives different results in two different environments, follow the steps below.
After migrations and upgrades, you may face issues caused by rudiments, such as those described below.
A training Business Process (BP) failure can be due to the following:
The execution mechanism for Work.AI is Bot Execution Platform (BEP), including for uploading models or moving them from environment to environment. Model artifacts and trained models are still stored in S3, but all meta information is maintained in the AutoML Model Management Service (AMMS).