9.3

Edit

Upgrade from v8.4.x to v9.3.x

Overview

The guide describes the steps for in-place SPA upgrade from 8.4.x to 9.3. This guide does not include the migration of Business Processes. For details about upgrading Business Processes, see migration notes for your SPA version.

caution

Do not change passwords when performing this upgrade. Use the same passwords from your SPA 8.4.x installation. Do not use dots . in paths.

Prerequisites

  • Workfusion 8.4.x product version is installed on Redhat 7.x.
  • Workfusion 9.3.x product install packages are copied and extracted to <PACKAGE_DIR> across the servers.
  • Collect required data from the previous installation.
  • Prepare Business Processes to upgrade BP migration notes.
  • Get the number of pages from the 8.4.x environment.

Predefined common variables are as follows:

  • WorkFusion user. Default is wfuser.
  • WorkFusion Home Directory. Default is /opt/workfusion.
  • WorkFusion Installer Directory. Default is /opt/workfusion/wf_installer.

Preparation

APP Server

  1. Stop services. The step prevents from changes in database during dumping. Run as <WF_USER>.

  2. Stop all at once.

    for i in $(alias | grep -o '\ .*-service') ; do eval "$i stop" ; done

  3. Stop services one by one.

    php-fpm-service stop
nginx-service stop
rpa_manager-service stop
tomcat-service stop
tomcat-nlp-service stop
tabula-service stop
wfagent-service stop

  4. Make sure there are no active processes.

    Example: how to stop all rpa_manager processes

    for i in `ps auxf | grep rpa_manager | awk '{ print $2}'`; do kill -9 $i; done

  5. Stop the Tabula service. Run as root.

    /etc/init.d/tabula stop

  6. Collect data from the Vault storage.

    • Download the 9.3.x installation package and use the recent loader-security-properties.jar from it.

    • Run as <WF_USER>.

    ## copy new loader-security-properties.jar to the current wf_sec_storage directory
cp <PACKAGE_DIR>/sources/loader-security-properties.jar <INSTALL_DIR>/apps/webapps/wf_sec_storage/loader-security-properties.jar
## go to the current wf_sec_storage directory
cd <INSTALL_DIR>/apps/webapps/wf_sec_storage/
## make a copy of connection properties
cp connectionProps.json connectionProps_customer_safe.json
## change Vault Safe to WFApp
sed -i s/WFInternal/WFApp/ connectionProps_customer_safe.json
## extract secrets 
<INSTALL_DIR>/lib/java/bin/java -jar loader-security-properties.jar wfagent connectionProps_customer_safe.json review > security-all.properties
    • Copy the security-all.properties file to DB server.

  7. Rename the installation 8.4.x directory. Run as root.

    mv <INSTALL_DIR> <INSTALL_DIR>_old

  8. Clear crontab jobs. Remove all Workfusion related cron jobs.

    Before deleting the jobs, make sure any non-Workfusion related cron jobs are backed up:

    crontab -l

    Run as root to delete all cron jobs.

    crontab -e

  9. Clean up WorkFusion init services. Run as root.

    cd /etc/init.d
rm -f rpa_manager tabula tomcat tomcat-nlp wfagent php-fpm nginx

PM Server

  1. Create dump. Run as <WF_USER>.

    mkdir <INSTALL_DIR>/dumps
<INSTALL_DIR>/lib/mongodb/mongo --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" admin
mongo.shell> db.grantRolesToUser("admin",["backup"])
mongo.shell> quit()
<INSTALL_DIR>/lib/mongodb/mongodump --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" --archive=<INSTALL_DIR>/dumps/mongo-dump.tar

  2. Stop services. Run as <WF_USER>.

    Stop all at once.

    for i in $(alias | grep -o '\ .*-service') ; do eval "$i stop" ; done

  3. Rename the installation 8.4.x directory. Run as root.

    mv <INSTALL_DIR> <INSTALL_DIR>_old

  4. Clean up WorkFusion init services. Run as root.

    cd /etc/init.d
rm -f mongod wfagent

OCR Server

  1. Stop services. Run as root.

    /etc/init.d/abbyy-fre11-licensing-service stop
/etc/init.d/wfagent stop
/etc/init.d/ocr2 stop

  2. Clean up WorkFusion init services. Run as root.

    cd /etc/init.d
rm -f abbyy-fre11-licensing-service wfagent ocr2

  3. Clean up ABBYY files. Run as root.

    cd /usr/local/bin
rm -rf ABBYY
cd /usr/local/lib
rm -rf ABBYY

  4. Clear crontab jobs. Remove all Workfusion related cron jobs.

    Before deleting the jobs, make sure any non-Workfusion related cron jobs are backed up:

    crontab -l

    Run as root to delete all cron jobs

    crontab -e

  5. Rename the installation 8.4.x directory. Run as root.

    mv <INSTALL_DIR> <INSTALL_DIR>_old

AutoML-Master Server

  1. Stop services. Run as <WF_USER>.

    wfmanager stop all

  2. Run as root

    service supervisord stop

  3. Clear crontab jobs. Run as <WF_USER> to delete all cron jobs.

    crontab -e

  4. Rename the installation 8.4.x directory. Run as root.

    mv <INSTALL_DIR> <INSTALL_DIR>_old

AutoML-Agent Server

  1. Stop services. Run as <WF_USER>.

    wfmanager stop all

  2. Run as root.

    service supervisord stop

  3. Rename the installation 8.4.x directory. Run as root.

mv <INSTALL_DIR> <INSTALL_DIR>_old

DB Server

  1. Convert usernames and secrets to YAML format. Run as <WF_USER>.

    cd <INSTALL_DIR>/app-all/

    source ./environment.conf
cat << EOF > secrets_override.yml

workfusion_db_user: $WORKFUSION_MYSQL_USER
workfusion_db_pass: $WORKFUSION_MYSQL_PASS
postgresql_admin_pass: $POSTGRES_PASS

workfusion_datastore_user: $WORKFUSION_PSQL_USER
workfusion_datastore_pass: $WORKFUSION_PSQL_PASS

rpa_manager_db_admin_user: $MONGO_ADMIN_NAME
rpa_manager_db_admin_pass: $MONGO_ADMIN_PASS

wfagent_master_db_user: $MONGO_ADMIN_NAME
wfagent_master_db_pass: $MONGO_ADMIN_PASS

wfagent_mongodb_ro_user: $MONGO_ADMIN_RO_NAME
wfagent_mongodb_ro_pass: $MONGO_ADMIN_RO_PASS

wfagent_username: $WFAGENT_AUTH_USER
wfagent_password: $WFAGENT_AUTH_PASSWORD

ocr_passwd: $OCR_PASSWD

wfbi_db_user: $WFBI_USER
wfbi_db_pass: $WFBI_PASSWORD

apm_hostname: $APMHOSTNAME
db_hostname: $DBHOSTNAME
app_hostname: $WFHOSTNAME
ocr_hostname: $OCRHOSTNAME
vds_master_hostname: $MLHOSTNAME
EOF

  2. Copy the resulting secrets_override.yml file to <PACKAGE_DIR>.

    cp secrets_override.yml <PACKAGE_DIR>/

Data backup

Create the dumps directory. Run as <WF_USER>:

mkdir -p <INSTALL_DIR>/dumps

MySQL

Run as <WF_USER>:

<INSTALL_DIR>/lib/mysql/bin/mysqldump --max_allowed_packet=1024M --add-drop-database --routines -S <INSTALL_DIR>/apps/mysql/mysql.sock -u root -p -B sqc -B wfdb  >> <INSTALL_DIR>/dumps/mysql-dump.sql

PostgreSQL

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<INSTALL_DIR>/lib/pgsql-9.4.7/lib
<INSTALL_DIR>/lib/pgsql-client-9.4.7/bin/pg_dumpall -U postgres -h 127.0.0.1 -p 5432 > <INSTALL_DIR>/dumps/postgres-all.sql

MongoDB

<INSTALL_DIR>/lib/mongodb/mongo --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" admin
mongo.shell> db.grantRolesToUser("admin",["backup"])
mongo.shell> quit()
<INSTALL_DIR>/lib/mongodb/mongodump --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" --archive=<INSTALL_DIR>/dumps/mongo-dump.tar

S3Emu

  1. Check how much space your S3 storage takes.

    <INSTALL_DIR>/lib/s3cmd/s3cmd -c <INSTALL_DIR>/apps/s3emu/etc/s3cfgfasttrack du -H | grep Total

  2. Ensure that you have enough disk space.

  3. Create a script that will back up S3 data.

    cat << 'EOF' > backup-s3.sh
#!/bin/bash
BUCKET_LIST=$(<INSTALL_DIR>/lib/s3cmd/s3cmd --signature-v2 -c <INSTALL_DIR>/apps/s3emu/etc/s3cfgfasttrack  ls | cut -d'/' -f3); \
for i in $BUCKET_LIST; do mkdir -p <INSTALL_DIR>/dumps/wf-s3-content/$i && \
<INSTALL_DIR>/lib/s3cmd/s3cmd -c <INSTALL_DIR>/apps/s3emu/etc/s3cfgfasttrack get -f -r s3://$i <INSTALL_DIR>/dumps/wf-s3-content/$i; done
EOF

  4. Make it executable and launch it.

    chmod +x backup-s3.sh
./backup-s3.sh

Copy old models

  1. Copy the old models file from s3://vds-models/models.prod to the dumps directory. Run as <WF_USER>:

    s3cmd sync s3://vds-models/models.flat-prod <INSTALL_DIR>/dumps/models.prod-8.4.x

    Nexus

    nexus stop
tar -czvf <INSTALL_DIR>/dumps/nexus.tar.gz <INSTALL_DIR>/apps/nexus/sonatype-work/nexus

  2. Stop services. Run as <WF_USER>.

  3. Stop all at once.

    for i in $(alias | grep -o '\ .*-service') ; do eval "$i stop" ; done

  4. Make sure there are no running processes.

  5. Rename the installation 8.4.x directory. Run as root:

    mv <INSTALL_DIR> <INSTALL_DIR>_old

Installation and data restoring

DB Server

For DB Server installation, see SPA installation for DB server.

Use the secrets_override.yml file to override default credentials.

./install.sh precheck db
./install.sh preinstall db
./install.sh install db -e @secrets_override.yml
./install.sh check db

S3Emu

Copy s3emu data to the MinIO data directory.

cp -r <INSTALL_DIR>_old/dumps/wf-s3-content/* <INSTALL_DIR>/minio/data/

MySQL

This need to be done only if new installation URL differs from the previous one. For example, if a port is changed or URL switched from HTTP to HTTPS. Example: old environment used port 80 non-TLS while the new one should work on port 443 with TLS: sed -i "s%http://wfapp.workfusion.com%https://wfapp.workfusion.com%g" mysql-dump.sql Example: old environment used port 443 while the new one should work on 8443 port: sed -i "s%https://wfapp.workfusion.com%https://wfapp.workfusion.com:8443%g" mysql-dump.sql

Upload the dump to MySQL.

mysql -u mysql -h 127.0.0.1 -p < mysql-dump.sql

PostgreSQL

  1. Connect to the PostgreSQL database.

    ## postgresadmin is the user postgresql_admin_user from secrets.yml 
psql -U postgresadmin -h 127.0.0.1

  2. Run the following queries in the PostgreSQL shell to remove connections that can prevent dropping of databases:

    SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'virtualizer_sandbox'
  AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'virtualizer'
  AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'wf_datastore'
  AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'wfml_automation'
  AND pid <> pg_backend_pid();

drop database virtualizer_sandbox;
drop database virtualizer;
drop database wf_datastore;
drop database wfml_automation;
\q

    This need to be done only if new installation URL differs from the previous one. For example, if port is changed or URL switched from HTTP to HTTPS. Example: old environment used port 80 non-TLS while the new one should work on port 443 with TLS: sed -i "s%http://wfapp.workfusion.com%https://wfapp.workfusion.com%g" postgres-all.sql Example: old environment used port 443 while the new one should work on 8443 port: sed -i "s%https://wfapp.workfusion.com%https://wfapp.workfusion.com:8443%g" postgres-all.sql

  3. Restore data from PostgreSQL 8.4.x dump:

    ## postgresadmin is the user postgresql_admin_user from secrets.yml 
psql -U postgresadmin -h 127.0.0.1 -f postgres-all.sql

MongoDB

Restore the database from backup.

mongo --host 127.0.0.1 --port 27016 --username admin --password "myadminpass" admin
mongo.shell> db.grantRolesToUser("admin",["root"])
mongorestore --host 127.0.0.1 --port 27016 --username admin --password "myadminpass" --archive=mongo-dump.tar

Nexus

wfmanager stop nexus

tar -xvzf <INSTALL_DIR>/dumps/nexus.tar.gz --strip 1 -C <INSTALL_DIR>/nexus/sonatype-work/

Vault

Upload the user secrets from 8.4.x Business Process to Vault.

## security-all.properties should be copied from APP server to DB
cp security-all.properties <INSTALL_DIR>/wf-sec-storage
cd <INSTALL_DIR>/wf-sec-storage
cp connectionProps_wf.json connectionProps_customer_safe.json
sed -i s/WFInternal/WFApp/ connectionProps_customer_safe.json
cat security-all.properties | grep -v 'This property is not configured.' |grep -v "It's all" | grep -v 'Key : Value' | grep -v 'You are working with safe' | grep -v 'Properties from secure storage' |sed s%\\[INFO\ \\]\ %%g | sed s/\ :\ /=/ > security-filtered-all.properties

source <INSTALL_DIR>/environment.sh
java -jar loader-security-properties.jar wfagent connectionProps_customer_safe.json security-filtered-all.properties

Final migrations

Run as <WF_USER>.

cd <PACKAGE_DIR>
./install.sh install db -e @secrets_override.yml
./install.sh check db -e @secrets_override.yml

APP Server

  1. Copy the WorkFusion license.

    cp <INSTALL_DIR>_old/apps/webapps/apache-tomcat-8.5.20/conf/license.properties <PACKAGE_DIR>/

  2. Follow SPA installation for APP server.

    • Run as root.
    cd <PACKAGE_DIR>
./install.sh precheck app
./install.sh preinstall app
    • Run as <WF_USER>.

    Use the secrets_override.yml file to override default credentials.

    cd <PACKAGE_DIR>
./install.sh install app -e @secrets_override.yml
./install.sh check app -e @secrets_override.yml

  3. Restart services.

    source <INSTALL_DIR>/environment.sh
wfmanager restart all

PM Server

  1. Follow SPA installation for PM server.

    • Run as root.
    cd <PACKAGE_DIR>
./install.sh precheck apm
./install.sh preinstall apm
    • Run as <WF_USER>.

    Use the secrets_override.yml file to override default credentials.

    cd <PACKAGE_DIR>
./install.sh install apm -e @secrets_override.yml
./install.sh check apm -e @secrets_override.yml

  2. Restore Mongo DB data.

    source <INSTALL_DIR>/environment.sh
mongorestore --host 127.0.0.1 --port 13017 --username mongodb --password "mongodb" --archive=<INSTALL_DIR>_old/dumps/mongo-dump.tar

Do not forget to transfer custom configuration to a new wfagent configuration directory.

OCR Server

Follow SPA installation for OCR server.

  • Run as root.

    cd <PACKAGE_DIR>
./install.sh precheck ocr
./install.sh preinstall ocr

  • Run as <WF_USER>.

    Use the secrets_override.yml file to override default credentials.

    cd <PACKAGE_DIR>
./install.sh install ocr -e @secrets_override.yml
./install.sh check ocr -e @secrets_override.yml

AutoML Master

  1. Follow SPA installation for AutoML Master.

    • Run as root.
    cd <PACKAGE_DIR>
./install.sh precheck vds-master -e @secrets_override.yml
./install.sh preinstall vds-master -e @secrets_override.yml
    • Run as <WF_USER>.

    Use the secrets_override.yml file to override default credentials.

    cd <PACKAGE_DIR>
./install.sh install vds-master -e @secrets_override.yml
./install.sh check vds-master -e @secrets_override.yml

  2. Copy the new models file from s3://vds-models/models.flat-prod to a temporary directory:

    <INSTALL_DIR>/tools/mc -C <INSTALL_DIR>/minio_client_conf cp s3/vds-models/models.flat-prod /tmp/

  3. Merge model files.

    cat <INSTALL_DIR>_old/dumps/models.prod-8.4.x >> /tmp/models.flat-prod

  4. Upload the merged file to s3://vds-models/models.flat-prod.

    <INSTALL_DIR>/tools/mc -C <INSTALL_DIR>/minio_client_conf cp /tmp/models.flat-prod s3/vds-models/models.flat-prod

  5. Restart wfmanager.

    source <INSTALL_DIR>/environment.sh
wfmanager restart all

AutoML Agent

Follow SPA installation for AutoML Agent.

  • Run as root.

    cd <PACKAGE_DIR>
./install.sh precheck vds-slave
./install.sh preinstall vds-slave

  • Run as <WF_USER>.

    Use the secrets_override.yml file to override default credentials.

    cd <PACKAGE_DIR>
./install.sh install vds-slave -e @secrets_override.yml
./install.sh check vds-slave -e @secrets_override.yml

Post-installation steps

Perform post-installation procedures as described in Post installation.

← PreviousNext →