Extend AutoML SDK with Python classifiers · WorkFusion Documentation

You can extend AutoML SDK with a set of Python classifiers based on Scikit-learn, MXNet, and so on. The extension allows using ML algorithms implemented in Python with AutoML pipelines.

Here's a list of supported Python classifiers:

Set up development environment

This section contains instructions for setting up a local development environment for proper building and running a custom model with Python support.

Install system packages

Install the required system packages (Python, libraries, compiler, and so on).

Windows

macOS

Linux

On a Windows machine, follow the steps:

Install Visual Studio Community Edition.
Install Python 2.7.16 or higher.

Remember to select the pip utility in the Customize Python window. After the setup, verify that the utility is installed.
```
<python_installed_dir>/python -m pip
```
If an error occurs, install pip:
1. Download get-pip.py with curl or via browser.
```
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
```
2. Install pip.
```
<PYTHON_INSTALLATION_DIR>/python get-pip.py
```

On a Mac, proceed with these steps:

Download and and install Python 2.7.16+
Install the Compiler (GNU Compiler Collection).
```
brew install gcc
```

On Linux, run the following commands:

Install Python on Fedora.

dnf install gcc python2 python-devel libstdc++

Install Python on Ubuntu.

apt-get install gcc python2.7 python-dev python-pip

Finally, add <python_installed_dir> to the PATH environment variable , if it's not there already.

Use virtual environment

If you want to use a virtual environment, set the environment variables:

AUTOML_PYTHON_BIN=<path_to_python_binary_file>

To set the variables, use the steps below, depending on your operating system:

MacOS: ~/.bash_profile → export <VARIABLE>=<VALUE>
Linux: ~/.bashrc → export <VARIABLE>=<VALUE>
Windows: My Computer > Properties / Environment variables → <VARIABLE>=<VALUE>

After that, quit and restart the terminal or your IDE.

note

If you're using Nvidia GPU with configured CUDA, or Intel CPU with the mkl driver, or both, you can install mxnet-cu92, or mxnet-mkl, or mxnet-cu92mkl instead of the mxnet package.

Install Python packages

Before proceeding with installing the Python packages, quit all instances of the terminal and start a new one or launch your IDE with an integrated terminal.

note

If you're planning to use a virtual environment, activate it before running the installation command below.

To install all required Python packages, run the following command:

python -m pip install --user pip==19.0.2 setuptools==40.8.0 wheel==0.33.0 mxnet==1.3.1 scikit-learn==0.20.2 mock==2.0.0

Implement Python classifiers

To implement a Python classifier, use the mlConfig Named Entity in your AutoML SDK configuration.

Here's an example of implementing RandomForestClassifier with default parameters:

@Named("mlConfig")
public Classifier mlConfig() {
    return RandomForestClassifier.builder()
            .build();
}

Here's an example of implementing RandomForestClassifier with custom parameters:

@Named("mlConfig")
public Classifier mlConfig() {
    return RandomForestClassifier.builder()
            .maxDepth(10)
            .estimators(100)
            .maxFeatures(MaxFeaturesType.LOG2)
            .build();
}

Here's an example of GenericPythonClassifier—a generic customizable classifier. It can be used to set up any classifier manually by configuring the parameters and Python model name directly.

@Named("mlConfig")
public Classifier mlConfig() {
    return GenericPythonClassifier.builder()
        .modelName("sklearn.RandomForestClassifier")
        .parameter("max_depth", 10)
        .parameter("n_estimators", 100)
        .parameter("random_state", 123)
        .parameter("verbose", true)
        .build()
}