Change AutoML settings in ZooKeeper
The AutoML Model Service is split into two smaller services:
AutoML Model Management Service manages all AutoML models, both model sources and trained models. This service also allows uploading, downloading, and getting (create, read, update, delete) information of model artifacts and trained models.
AutoML Model Service manages all AutoML operations, such as training (start, stop, check status) and execution.
AutoML Gateway Service parameters
AutoML Gateway Service routes all requests from client systems to internal systems.
| Parameter | Description | Required | Default value |
|---|---|---|---|
automl.service.gateway.external-host | External host and port when application is used under proxy. Example: some-host.workfusion.com:8000 | Yes | localhost:8080 |
automl.service.gateway.base-path | Base path is used to define version of services. | Yes | /v1 |
automl.service.model.url | URL to internal AutoML Model Service. Example: http://localhost:7052/automl-model-service | Yes | http://localhost:8081 |
automl.service.model.management.url | URL to AutoML Model Management Service. | Yes | http://localhost:8082 |
automl.service.model.management.base-path | Base path for AutoML Model Management Service endpoint. | Yes | /v2 |
AutoML Model Service parameters
| Parameter | Description | Required | Default value |
|---|---|---|---|
server.port | Application port. | Yes | |
server.servlet.contextPath | Application servlet context path prefix. | Yes | /automl-model-service |
server.address | Application address. | Yes | 127.0.0.1 |
spring.cloud.zookeeper.enabled | Enable or disable the application configuration via the ZooKeeper cloud config. | No | true |
spring.cloud.zookeeper.connect-string | ZooKeeper connection string. Example: some-host.workfusion.com:2181 | No |
The following parameters need to be configured in ZooKeeper under the corresponding application name node. By default, /config/automl-model-service/ for AutoML model service.
| Parameter | Description | Required | Default value |
|---|---|---|---|
automl.service.model.external-host | External host and port when application is used under proxy. Example: some-host.workfusion.com:8000 | No | |
automl.service.model.base-path | Base path is used to define version of services. | Yes | /v1 |
automl.service.model.base-dir | Base directory for persisting service data. | Yes | ./data |
automl.service.model.management.url | URL to AutoML Model Management Service. | Yes | http://localhost:8082 |
automl.service.model.management.base-path | Base path to AutoML Model Management Service endpoint. | No | /v2 |
spring.profiles.include | Active spring profiles for application. | No | execution-client |
automl.service.model.training.timeout | Timeout for model training starting from sending task till receiving the result. Supports an expression for java.util.Duration. | No | 12h |
automl.service.model.execution.operation.total-timeout | Timeout for execution operations starting from sending task till receiving the result. Supports an expression for java.util.Duration. | Yes | 2m |
automl.service.model.execution.prepare.shared-dir | Path to shared NFS folder to prepare data for execution. | Yes | /tempDir/ |
automl.service.model.execution.worker.profile | Worker profile name for execution. | No | execution |
execution.app.id | Application name that starts the execution operation, Execution Framework parameter. | No | execution-default-app |
automl.service.model.env | Environment name. Used to look up model artifacts on S3. | Yes | local |
automl.service.model.training.limit.total | Limit for simultaneously running model trainings. | No | 8 |
automl.service.model.base-dir | Base directory for persisting service data | Yes | ./data |
automl.service.model.cluster.username | Username used for running training workers on cluster. | Yes | wfuser |
automl.service.model.cluster.work-dir | Working folder path on Mesos. Example: /opt/workfusion/some-folder/ | Yes | /data/vds/eval/ |
automl.service.model.s3.bucket.models | Bucket name with model artifacts on S3. | Yes | |
automl.service.model.s3.bucket.assets | Bucket name with trained models and other resources on S3. | Yes | |
automl.service.model.s3.bucket.worker.execution.models | Bucket name with trained models for BEP execution operations (may be the same as assets bucket name). | Yes | |
s3.endpointUrl | URL for S3 connection. Example: https://some-s3.workfusion.com:443 | Yes | |
s3.accessKey | Authentication access key for S3. | Yes | |
s3.secretKey | Authentication secret key for S3. | Yes | |
automl.service.model.aws.useAwsCli | Use AWS command line. | No | false |
automl.service.model.schedule.update-timeout-training-fixed-delay | Interval for model training timeout check. Supports java.util.Duration. | No | 15m |
The following parameters need to be configured in ZooKeeper under global application parameters node. Same for all BEP applications: /config/application/.
| Parameter | Description | Required | Default value |
|---|---|---|---|
spring.rabbitmq.host | RabbitMQ host. | Yes | |
spring.rabbitmq.username | Authentication username for RabbitMQ host. | Yes | |
spring.rabbitmq.password | Authentication password for RabbitMQ host. | Yes | |
spring.rabbitmq.virtual-host | RabbitMQ virtual host. | Yes |
AutoML Model Management Service parameters
| Parameter | Description | Required | Default value |
|---|---|---|---|
server.port | Application port. | Yes | |
server.servlet.contextPath | Application servlet context path prefix. | Yes | /automl-model-management-service |
server.address | Application address. | Yes | 127.0.0.1 |
spring.cloud.zookeeper.enabled | Enable or disable the application configuration via the ZooKeeper cloud config. | No | true |
spring.cloud.zookeeper.connect-string | ZooKeeper connection string. Example: some-host.workfusion.com:2181 | No |
The following parameters need to be configured in ZooKeeper under the corresponding application name node. By default, /config/automl-model-management-service/ for AutoML Model Management Service.
| Parameter | Description | Required | Default value |
|---|---|---|---|
automl.service.model.external-host | External host and port when application is used under proxy. Example: some-host.workfusion.com:8000 | No | |
automl.service.model.management.base-path | Base path to AutoML Model Management Service endpoint. | Yes | /v2 |
automl.service.model.management.env | Environment name used for look-up model artifacts on S3. | Yes | local |
automl.service.model.management.s3.bucket.models | Bucket name with model artifacts on S3. | No | vds-resources |
automl.service.model.management.s3.bucket.assets | Bucket name with trained models for BE. | No | vds-models |
automl.service.model.management.s3.bucket.import | Bucket name with packages for import. May be the same as export bucket name. | No | import |
automl.service.model.management.s3.bucket.export | Bucket name with exported artifacts or models. May be the same as import bucket name. | No | export |
s3.endpointUrl | URL for S3 connection. Example: https://some-s3.workfusion.com:443. | Yes | |
s3.accessKey | Authentication access key for S3. | Yes | |
s3.secretKey | Authentication secret key for S3. | Yes | |
automl.service.model.schedule.update-resources-fixed-delay | Interval for updating resources from S3. Supports java.util.Duration. | No | 5m |
nexus.endpoint | Endpoint to Nexus instance. | Yes | http://localhost:8081/nexus |
nexus.username | Nexus username. | Yes | admin |
nexus.password | Nexus password. | Yes | admin123 |
nexus.repository | Nexus repository for workers deployment. | Yes | Workers |
spring.jpa.datasource.url | URL to database with AutoML MMS schema. | Yes | JDBC: sqlserver://localhost:1433;databaseName=workfusion;schema=automl_mms |