Architecture components
Component |
Description |
|---|---|
Gateway service |
WorkFusion-created set of REST services to provide control gate for AutoML features exposure for higher-level applications (Control Tower). |
RabbitMQ |
RabbitMQ is a messaging broker: an intermediary for messaging. Gateway services communicates with other services via RabbitMQ. |
S3 File Storage |
Is used to store: binary model, results of training and training set. |
Mesos adapter |
Service which is responsible for the following operations: submit training and check evaluation status. |
Marathon |
Keeps info about models configuration and info about resources required to start model. |
Mesos |
A cluster management system Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications (foe example, Hadoop, Spark, Kafka, Elasticsearch) with API’s for resource management and scheduling across entire datacenter and cloud environments. Mesos consists of a master daemon that manages agent daemons running on each cluster node, and Mesos frameworks that run tasks on these agents. Mesos Master A Mesos master aggregates resource offers from all agent nodes and provides them to registered frameworks. Mesos Agent A Mesos agent node runs a discrete Mesos task on behalf of a framework. It is an agent instance registered with the Mesos master. The synonym of agent node is worker or slave node. |
ZooKeeper |
Storage for settings. |
Scaling service |
Scaling service is responsible for serving list of available models to Gateway service and launching models. |
ML DB |
DB to store evaluation results known as Tableau DB. |
service-wfbi |
Rest Service which is used to communicate with ML DB. |