Skip to main content
Version: 10.2.4

Extend AutoML SDK with Python classifiers

You can extend AutoML SDK with a set of Python classifiers based on Scikit-learn, MXNet, and so on. The extension allows using ML algorithms implemented in Python with AutoML pipelines.

Here's a list of supported Python classifiers:

Set up development environment​

This section contains instructions for setting up a local development environment for proper building and running a custom model with Python support.

Install system packages​

Install the required system packages (Python, libraries, compiler, and so on).

{/DOCUSAURUS_CODE_TABS``/} {/Windows/}

On a Windows machine, follow the steps:

  1. Install Visual Studio Community Edition.

  2. Download and install Python 3.7 or Python 3.10.

    Remember to select the pip utility in the Customize Python window. After the setup, verify that the utility is installed.

    <python_installed_dir>/python -m pip

    If an error occurs, install pip:

    1. Download get-pip.py with curl or via browser.

      curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

    2. Install pip.

      <PYTHON_INSTALLATION_DIR>/python get-pip.py

{/macOS/}

On a Mac, proceed with these steps:

  1. Download and and install Python 2.7.16+.

  2. Install the Compiler (GNU Compiler Collection).

    brew install gcc

{/Linux/}

On Linux, run the following commands:

  1. Install Python on Fedora.
    dnf install gcc python2 python-devel libstdc++
  2. Install Python on Ubuntu.
    apt-get install gcc python2.7 python-dev python-pip

{/END_DOCUSAURUS_CODE_TABS``/}

Finally, add <python_installed_dir> to the PATH environment variable , if it's not there already.

Install Python packages​

Before proceeding with installing the Python packages, quit all instances of the terminal and start a new one or launch your IDE with an integrated terminal.

note

If you're planning to use a virtual environment, activate it before implementing Python classifiers.

To install all required Python packages, run the following command:

python -m pip install --user pip==19.0.2 setuptools==40.8.0 wheel==0.33.0 mxnet==1.3.1 scikit-learn==0.20.2 mock==2.0.0

Use virtual environment​

If you want to use a virtual environment, set the environment variables:

AUTOML_PYTHON_BIN=<path_to_python_binary_file>

To set the variables, use the steps below, depending on your operating system:

  • MacOS: ~/.bash_profile → export <VARIABLE>=<VALUE>
  • Linux: ~/.bashrc → export <VARIABLE>=<VALUE>
  • Windows: My Computer > Properties / Environment variables → <VARIABLE>=<VALUE>

After that, quit and restart the terminal or your IDE.

note

If you're using Nvidia GPU with configured CUDA, or Intel CPU with the mkl driver, or both, you can install mxnet-cu92, or mxnet-mkl, or mxnet-cu92mkl instead of the mxnet package.

Implement Python classifiers​

To implement a Python classifier, use the mlConfig Named Entity in your AutoML SDK configuration.

Here's an example of implementing RandomForestClassifier with default parameters:

@Named("mlConfig")
public Classifier mlConfig() {
    return RandomForestClassifier.builder()
            .build();
}

Here's an example of implementing RandomForestClassifier with custom parameters:

@Named("mlConfig")
public Classifier mlConfig() {
    return RandomForestClassifier.builder()
            .maxDepth(10)
            .estimators(100)
            .maxFeatures(MaxFeaturesType.LOG2)
            .build();
}

Here's an example of GenericPythonClassifier: a generic customizable classifier. It can be used to set up any classifier manually by configuring the parameters and Python model name directly.

@Named("mlConfig")
public Classifier mlConfig() {
    return GenericPythonClassifier.builder()
        .modelName("sklearn.RandomForestClassifier")
        .parameter("max_depth", 10)
        .parameter("n_estimators", 100)
        .parameter("random_state", 123)
        .parameter("verbose", true)
        .build()
}