Hardware requirements
WorkFusion AutoML models are generated based on HPO (Hyper Parameter Optimization) framework:
- There are 1K+ of components with different parameters (Feature-Extractors, Annotations, ML algorithms, and so on).
- Model is a subset of these components.
- During the training process, HPO finds the best subset of components or parameters based on a provided data set.
- Each field is represented by a separate model. If there are 10 fields, 10 models are trained independently and combined into one multi model.
Once the best subset of the components is found, you can fix the AutoML model configuration, and during the next re-training, HPO will not be executed.
The infrastructure requirements are as follows:
| Installation | Components | CPU | RAM (GB) | HDD (Addressable space after formatting) |
|---|---|---|---|---|
| AutoML Cluster | Master, NFS | 8 | 32 | 250 GB |
| NFS share | - | - | 2 TB | |
| 3x Agents | 8 | 32 | 250 GB |
Training
During training, multiple experiments are running to define the best feature combination.
Memory: 96 * ParallelFields
CPU: 24 * ParallelFields
Execution
During extraction, there is already a trained model. Therefore, extraction requires less resources than training.
- Memory: 8 * ParallelModels
- CPU: 2 * ParallelModels
Requirements
| Server | Training fields simultaneously | Execute models simultaneously |
|---|---|---|
| RAM: 128, CPU: 32 | 1 | 12 |
| RAM: 256, CPU: 64 | 3 | 28 |