Edit

Deploy Python environment and configure IA Cloud to use it

To enable working with a custom Python environment, follow the instruction:

1. Add your Python version (3.7) to a directory, for example, /opt/workfusion/python/python3.7.9.

2. Install dependencies as shown below:

   ls <dir_with_whl> > reqs.txt
   PYTHONPATH="" /opt/workfusion/python/python3.7./bin/python3 -m pip install-r reqs.txt -f <dir_with_whl> --no-deps --no-cache-dir
   

3. On each Agent, edit the /opt/workfusion/environment.sh file by changing PYTHONPATH=... to PYTHONPATH=$PYTHONPATH: ....

4. In ZooKeeper on the Master node, reconfigure Workers to use the new Python version:

   /opt/workfusion/zookeeper/bin/zkCli.sh set /config/worker-management-service/bep.wms.worker.execute.env-variables "{'PYTHONPATH': '/opt/workfusion/python/python3.7.9/lib/python37.zip:/opt/workfusion/python/python3.7.9/lib/python3.7:/opt/workfusion/python/python3.7.9/lib/python3.7/lib-dynload:/opt/workfusion/python/python3.7.9/lib/python3.7/site-packages', 'AUTOML_PYTHON_BIN': '/opt/workfusion/python/python3.7.9/bin/python3', 'LD_LIBRARY_PATH': '/opt/workfusion/python/python3.7.9/lib:/opt/workfusion/mesos/usr/lib/:/opt/workfusion/mesos/usr/lib64/:/opt/workfusion/vowpal-wabbit/lib:/opt/workfusion/rpms/usr/lib:/opt/workfusion/rpms/usr/lib64:/opt/workfusion/rpms/lib64/openssl/engines:/opt/workfusion/rpms/libexec/coreutils:/opt/workfusion/mesos/usr/lib/:/opt/workfusion/mesos/usr/lib64/:/opt/workfusion/vowpal-wabbit/lib:/opt/workfusion/rpms/usr/lib:/opt/workfusion/rpms/usr/lib64:/opt/workfusion/rpms/lib64/openssl/engines:/opt/workfusion/rpms/libexec/coreutils', 'MXNET_USE_OPERATOR_TUNING': '0'}"
   

5. Restart Worker Management Service.

6. Restart Workers.