Build custom Python model

The recommended best practice is to develop a Python model package as a pure Python project, with tests and so on, and to use it as a dependency in an Archetype-generated project intended for integration with the WorkFusion ecosystem.

Build data preparation

If your Python installation contains not all required libraries, you can pack the missing ones as wheels and add them to your model. For that, proceed in either of the two ways:

• Add the dependendencies to src/main/python/.meta/packages, formatting them as in the requirements.txt for pip in Python. In this case, the wheels are built automatically as you build a hyper model with Maven.

• Use the deps_builder.sh script and add the wheels to src/main/python/deps/wheels. The wheels end up inside a hyper model and are installed at Worker launch.

Build project

Once the model is ready, build your project by running the following command in Maven: mvn clean install.

The action packs all Python sources into a JAR file to be executed on the cluster and creates a model artifact (Worker). You can find the resulting model artifact (Worker) in the ./deploy directory inside the /%code%/%version sub-directories.

note

Whenever you change the Python part of your model, rebuild it before training or execution using the mvn clean install command. This is essential to get the Python changes applied during its run.