9.2

Edit

Upgrade from v8.4.x to v9.2.x

Overview

This guide describes the steps for in-place SPA upgrade from 8.4.x to 9.2. This guide does not include the migration of business processes. For details about upgrading business processes, see the migration notes for your version of SPA.

caution

Do not change passwords when performing this upgrade. Use the same passwords from your SPA 8.4.x installation.

Do not use dots "." in paths

Prerequisites

  • Workfusion 8.4.x product version is installed on Redhat 7.x.
  • Workfusion 9.2.x product install packages are copied and extracted to <PACKAGE_DIR> across the servers.
  • Collect required data from previous installation.
  • Prepare Business Processes to upgrade BP migration notes.
  • Get the the number of pages from the 8.4.x environment.

Predefined common variables:

  • WF_USER: WorkFusion user. Default is wfuser.
  • INSTALL_DIR: WorkFusion Home Directory. Default is /opt/workfusion.
  • <PACKAGE_DIR>: WorkFusion Installer Directory. Default is /opt/workfusion/wf_installer

Preparation

APP Server

  1. Stop services.

    This step prevents from changes in database during dumping.

    Run as <WF_USER>.

    Stop all at once:

    for i in $(alias | grep -o '\ .*-service') ; do eval "$i stop" ; done

    Stop services one by one:

    php-fpm-service stop
nginx-service stop
rpa_manager-service stop
tomcat-service stop
tomcat-nlp-service stop
tabula-service stop
wfagent-service stop

    Make sure there are no any active processes

    Example: how to stop all rpa_manager processes

    for i in `ps auxf | grep rpa_manager | awk '{ print $2}'`; do kill -9 $i; done

  2. Stop the Tabula service. Run as root.

    /etc/init.d/tabula stop

  3. Collect data from the Vault storage.

    • Download the 9.2.x installation package and use the recent loader-security-properties.jar from it.

    • Run as <WF_USER>.

        ## copy new loader-security-properties.jar to the current wf_sec_storage directory
    cp <PACKAGE_DIR>/sources/loader-security-properties.jar <INSTALL_DIR>/apps/webapps/wf_sec_storage/loader-security-properties.jar
    ## go to the current wf_sec_storage directory
    cd <INSTALL_DIR>/apps/webapps/wf_sec_storage/
    ## make a copy of connection properties
    cp connectionProps.json connectionProps_customer_safe.json
    ## change Vault Safe to WFApp
    sed -i s/WFInternal/WFApp/ connectionProps_customer_safe.json
    ## extract secrets 
    <INSTALL_DIR>/lib/java/bin/java -jar loader-security-properties.jar wfagent connectionProps_customer_safe.json review > security-all.properties
    • Copy the security-all.properties file to DB server.

  4. Rename the 8.4.x installation directory. Run as root.

    mv <INSTALL_DIR> <INSTALL_DIR>_old

  5. Remove all WorkFusion-related cron jobs. Before deleting the jobs, make sure any non-Workfusion related cron jobs are backed up:

    crontab -l

    Run as root to delete all cron jobs.

    crontab -e

  6. Clean up the WorkFusion init services. Run as root.

    cd /etc/init.d
rm -f rpa_manager tabula tomcat tomcat-nlp wfagent php-fpm nginx

PM Server

  1. Create dump. Run as <WF_USER>.

    mkdir <INSTALL_DIR>/dumps
<INSTALL_DIR>/lib/mongodb/mongo --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" admin
mongo.shell> db.grantRolesToUser("admin",["backup"])
mongo.shell> quit()
<INSTALL_DIR>/lib/mongodb/mongodump --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" --archive=<INSTALL_DIR>/dumps/mongo-dump.tar

  2. Stop services. Run as <WF_USER>.

    Stop all at once.

    for i in $(alias | grep -o '\ .*-service') ; do eval "$i stop" ; done

  3. Rename the 8.4.x installation directory. Run as root.

    mv <INSTALL_DIR> <INSTALL_DIR>_old

  4. Clean up the WorkFusion init services. Run as root.

    cd /etc/init.d
rm -f mongod wfagent

OCR Server

  1. Stop services. Run as root.

    /etc/init.d/abbyy-fre11-licensing-service stop
/etc/init.d/wfagent stop
/etc/init.d/ocr2 stop

  2. Clean up the WorkFusion init services. Run as root.

    cd /etc/init.d
rm -f abbyy-fre11-licensing-service wfagent ocr2

  3. Clean up ABBYY files. Run as root.

    cd /usr/local/bin
rm -rf ABBYY
cd /usr/local/lib
rm -rf ABBYY

  4. Remove all Workfusion related cron jobs. Before deleting the jobs, make sure any non-Workfusion related cron jobs are backed up:

    crontab -l

    Run as root to delete all cron jobs.

    crontab -e

  5. Rename the 8.4.x installation directory. Run as root.

    mv <INSTALL_DIR> <INSTALL_DIR>_old

AutoML-Master Server

  1. Stop services. Run as <WF_USER>.

    wfmanager stop all

    Run as root

    service supervisord stop

  2. Clear crontab jobs. Run as <WF_USER> to delete all cron jobs.

    crontab -e

  3. Rename the 8.4.x installation directory. Run as root.

    mv <INSTALL_DIR> <INSTALL_DIR>_old

AutoML-Agent Server

  1. Stop services. Run as <WF_USER>.

    wfmanager stop all

    Run as root.

    service supervisord stop

  2. Rename the 8.4.x installation directory. Run as root.

    mv <INSTALL_DIR> <INSTALL_DIR>_old

DB Server

  1. Convert usernames and secrets to the YAML format. Run as <WF_USER>.

    cd <INSTALL_DIR>/app-all/

    source ./environment.conf
cat << EOF > secrets_override.yml

workfusion_db_user: $WORKFUSION_MYSQL_USER
workfusion_db_pass: $WORKFUSION_MYSQL_PASS
postgresql_admin_pass: $POSTGRES_PASS

workfusion_datastore_user: $WORKFUSION_PSQL_USER
workfusion_datastore_pass: $WORKFUSION_PSQL_PASS

rpa_manager_db_admin_user: $MONGO_ADMIN_NAME
rpa_manager_db_admin_pass: $MONGO_ADMIN_PASS

wfagent_master_db_user: $MONGO_ADMIN_NAME
wfagent_master_db_pass: $MONGO_ADMIN_PASS

wfagent_mongodb_ro_user: $MONGO_ADMIN_RO_NAME
wfagent_mongodb_ro_pass: $MONGO_ADMIN_RO_PASS

wfagent_username: $WFAGENT_AUTH_USER
wfagent_password: $WFAGENT_AUTH_PASSWORD

ocr_passwd: $OCR_PASSWD

wfbi_db_user: $WFBI_USER
wfbi_db_pass: $WFBI_PASSWORD

apm_hostname: $APMHOSTNAME
db_hostname: $DBHOSTNAME
app_hostname: $WFHOSTNAME
ocr_hostname: $OCRHOSTNAME
vds_master_hostname: $MLHOSTNAME
EOF

  2. Copy the resulting secrets_override.yml file to <PACKAGE_DIR>.

    cp secrets_override.yml <PACKAGE_DIR>/

Data backup

Create the dumps directory. Run as <WF_USER>.

mkdir -p <INSTALL_DIR>/dumps

MySQL

Run as <WF_USER>.

<INSTALL_DIR>/lib/mysql/bin/mysqldump --max_allowed_packet=1024M --add-drop-database --routines -S <INSTALL_DIR>/apps/mysql/mysql.sock -u root -p -B sqc -B wfdb  >> <INSTALL_DIR>/dumps/mysql-dump.sql

PostgreSQL

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<INSTALL_DIR>/lib/pgsql-9.4.7/lib
<INSTALL_DIR>/lib/pgsql-client-9.4.7/bin/pg_dumpall -U postgres -h 127.0.0.1 -p 5432 > <INSTALL_DIR>/dumps/postgres-all.sql

MongoDB

<INSTALL_DIR>/lib/mongodb/mongo --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" admin
mongo.shell> db.grantRolesToUser("admin",["backup"])
mongo.shell> quit()
<INSTALL_DIR>/lib/mongodb/mongodump --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" --archive=<INSTALL_DIR>/dumps/mongo-dump.tar

S3Emu

  1. Check how much space your S3 storage takes.

    <INSTALL_DIR>/lib/s3cmd/s3cmd -c <INSTALL_DIR>/apps/s3emu/etc/s3cfgfasttrack du -H | grep Total

  2. Ensure that you have enough disk space.

  3. Create a script that will back up S3 data.

    cat << 'EOF' > backup-s3.sh
#!/bin/bash
BUCKET_LIST=$(<INSTALL_DIR>/lib/s3cmd/s3cmd --signature-v2 -c <INSTALL_DIR>/apps/s3emu/etc/s3cfgfasttrack  ls | cut -d'/' -f3); \
for i in $BUCKET_LIST; do mkdir -p <INSTALL_DIR>/dumps/wf-s3-content/$i && \
<INSTALL_DIR>/lib/s3cmd/s3cmd -c <INSTALL_DIR>/apps/s3emu/etc/s3cfgfasttrack get -f -r s3://$i <INSTALL_DIR>/dumps/wf-s3-content/$i; done
EOF

  4. Make it executable and launch it.

    chmod +x backup-s3.sh
./backup-s3.sh

  5. Copy the old models file from s3://vds-models/models.prod to the dumps directory. Run as <WF_USER>.

    s3cmd sync s3://vds-models/models.flat-prod <INSTALL_DIR>/dumps/models.prod-8.4.x

Nexus

nexus stop
tar -czvf <INSTALL_DIR>/dumps/nexus.tar.gz <INSTALL_DIR>/apps/nexus/sonatype-work/nexus

Stop services

Run as <WF_USER>

Stop all at once

for i in $(alias | grep -o '\ .*-service') ; do eval "$i stop" ; done

Make sure there are no running processes launched from. Rename the 8.4.x installation directory

Run as root.

mv <INSTALL_DIR> <INSTALL_DIR>_old

Installation and data restoring

DB Server

Follow SPA installation for DB server.

Use secrets_override.yml file to override default credentials.

./install.sh precheck db
./install.sh preinstall db
./install.sh install db -e @secrets_override.yml
./install.sh check db

S3Emu

Copy s3emu data to the MinIO data directory.

cp -r <INSTALL_DIR>_old/dumps/wf-s3-content/* <INSTALL_DIR>/minio/data/

MySQL

This need to be done only if new installation URL differs from previous, e.g. if port is changed or URL switched from http to https.

Example: old environment used port 80 non-TLS while the new one should work on port 443 with TLS sed -i "s%http://wfapp.workfusion.com%https://wfapp.workfusion.com%g" mysql-dump.sql

Example: old environment used port 443 while the new one should work on 8443 port sed -i "s%https://wfapp.workfusion.com%https://wfapp.workfusion.com:8443%g" mysql-dump.sql

Upload the dump to MySQL.

mysql -u mysql -h 127.0.0.1 -p < mysql-dump.sql

PostgreSQL

  1. Connect to PostgreSQL database.

    ## postgresadmin is the user postgresql_admin_user from secrets.yml 
psql -U postgresadmin -h 127.0.0.1

  2. Run the following queries in the PostgreSQL shell to remove connections that may prevent dropping of databases:

    SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'virtualizer_sandbox'
  AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'virtualizer'
  AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'wf_datastore'
  AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'wfml_automation'
  AND pid <> pg_backend_pid();

drop database virtualizer_sandbox;
drop database virtualizer;
drop database wf_datastore;
drop database wfml_automation;
\q

    This need to be done only if new installation URL differs from previous, e.g. if port is changed or URL switched from http to https.

    Example: old environment used port 80 non-TLS while the new one should work on port 443 with TLS sed -i "s%http://wfapp.workfusion.com%https://wfapp.workfusion.com%g" postgres-all.sql

    Example: old environment used port 443 while the new one should work on 8443 port sed -i "s%https://wfapp.workfusion.com%https://wfapp.workfusion.com:8443%g" postgres-all.sql

  3. Restore data from PostgreSQL 8.4.x dump:

## postgresadmin is the user postgresql_admin_user from secrets.yml 
psql -U postgresadmin -h 127.0.0.1 -f postgres-all.sql

MongoDB

Restore the database from backup.

mongo --host 127.0.0.1 --port 27016 --username admin --password "myadminpass" admin
mongo.shell> db.grantRolesToUser("admin",["root"])
mongorestore --host 127.0.0.1 --port 27016 --username admin --password "myadminpass" --archive=mongo-dump.tar

Nexus

wfmanager stop nexus

tar -xvzf <INSTALL_DIR>/dumps/nexus.tar.gz --strip 1 -C <INSTALL_DIR>/nexus/sonatype-work/

Vault

Upload the user secrets from 8.4.x Business Process to Vault.

## security-all.properties should be copied from APP server to DB
cp security-all.properties <INSTALL_DIR>/wf-sec-storage
cd <INSTALL_DIR>/wf-sec-storage
cp connectionProps_wf.json connectionProps_customer_safe.json
sed -i s/WFInternal/WFApp/ connectionProps_customer_safe.json
cat security-all.properties | grep -v 'This property is not configured.' |grep -v "It's all" | grep -v 'Key : Value' | grep -v 'You are working with safe' | grep -v 'Properties from secure storage' |sed s%\\[INFO\ \\]\ %%g | sed s/\ :\ /=/ > security-filtered-all.properties

source <INSTALL_DIR>/environment.sh
java -jar loader-security-properties.jar wfagent connectionProps_customer_safe.json security-filtered-all.properties

Final migrations

Run as <WF_USER>.

cd <PACKAGE_DIR>
./install.sh install db -e @secrets_override.yml
./install.sh check db -e @secrets_override.yml

APP Server

Copy the WorkFusion license to:

cp <INSTALL_DIR>_old/apps/webapps/apache-tomcat-8.5.20/conf/license.properties <PACKAGE_DIR>/

Follow the SPA installation article. for APP server

  • Run as root

    cd <PACKAGE_DIR>
./install.sh precheck app
./install.sh preinstall app

  • Run as <WF_USER>.

    Use secrets_override.yml file to override default credentials.

cd <PACKAGE_DIR>
./install.sh install app -e @secrets_override.yml
./install.sh check app -e @secrets_override.yml

Restart services

source <INSTALL_DIR>/environment.sh
wfmanager restart all

PM Server

Follow SPA installation for PM server.

  • Run as root

    cd <PACKAGE_DIR>
./install.sh precheck apm
./install.sh preinstall apm

  • Run as <WF_USER>.

    Use secrets_override.yml file to override default credentials.

    cd <PACKAGE_DIR>
./install.sh install apm -e @secrets_override.yml
./install.sh check apm -e @secrets_override.yml

Restore Mongo DB data.

source <INSTALL_DIR>/environment.sh
mongorestore --host 127.0.0.1 --port 13017 --username mongodb --password "mongodb" --archive=<INSTALL_DIR>_old/dumps/mongo-dump.tar

Do not forget to transfer custom configuration to a new wfagent configuration directory.

OCR Server

Follow SPA installation for OCR server.

  • Run as root.

    cd <PACKAGE_DIR>
./install.sh precheck ocr
./install.sh preinstall ocr

  • Run as <WF_USER>.

    Use secrets_override.yml file to override default credentials.

    cd <PACKAGE_DIR>
./install.sh install ocr -e @secrets_override.yml
./install.sh check ocr -e @secrets_override.yml

AutoML Master

Follow SPA installation for AutoML Master.

  • Run as root.

    cd <PACKAGE_DIR>
./install.sh precheck vds-master -e @secrets_override.yml
./install.sh preinstall vds-master -e @secrets_override.yml

  • Run as <WF_USER>.

    Use secrets_override.yml file to override default credentials.

    cd <PACKAGE_DIR>
./install.sh install vds-master -e @secrets_override.yml
./install.sh check vds-master -e @secrets_override.yml

Copy new models file from s3://vds-models/models.flat-prod to a temporary directory:

<INSTALL_DIR>/tools/mc -C <INSTALL_DIR>/minio_client_conf cp s3/vds-models/models.flat-prod /tmp/

Merge models files.

cat <INSTALL_DIR>_old/dumps/models.prod-8.4.x >> /tmp/models.flat-prod

Upload the merged file to s3://vds-models/models.flat-prod.

<INSTALL_DIR>/tools/mc -C <INSTALL_DIR>/minio_client_conf cp /tmp/models.flat-prod s3/vds-models/models.flat-prod

Restart wfmanager.

source <INSTALL_DIR>/environment.sh
wfmanager restart all

AutoML Agent

Follow SPA installation for AutoML Agent.

  • Run as root.

    cd <PACKAGE_DIR>
./install.sh precheck vds-slave
./install.sh preinstall vds-slave

  • Run as <WF_USER>

    Use secrets_override.yml file to override default credentials.

    cd <PACKAGE_DIR>
./install.sh install vds-slave -e @secrets_override.yml
./install.sh check vds-slave -e @secrets_override.yml

Post-installation steps

Perform post-installation procedures as described in Post-installation.

← PreviousNext →