9.0

Edit

Upgrade from 8.4.x to 9.0.x

Prerequisites

Root access required to successfully accomplish several steps.

You have workfusion product 8.4 installed on Redhat 7. You downloaded 9.0.x installer to all servers that will be upgraded.

This is a guide how to do an in-place upgrade from 8.4 non-TLS to 9.0.x non-TLS.

Collect required data from previous installation.

SSH into APP server.

Preparation

General steps

  1. Stop running services on APP server (except nginx because It will be needed for making backups of S3 buckets).

    This step prevents from changes in database during dumping.

    Run as wfuser on APP server:

    php-fpm-service stop
rpa_manager-service stop
tomcat-service stop
tomcat-nlp-service stop
tabula-service stop
wfagent-service stop

  2. Stop RPA Manager. Run as root on APP server:

    for i in `ps auxf | grep rpa_manager | awk '{ print $2}'`; do kill -9 $i; done

  3. Stop Tabula. Run as root on APP server:

    /etc/init.d/tabula stop

  4. Collect data from the Vault storage:

    1. Download the 9.0.x installation package and use the recent loader-security-properties-8.4.5.jar file from it.

    2. Run as wfuser on APP server:

    cp workfusion-9.0.x/sources/loader-security-properties-8.4.5.jar /opt/workfusion/apps/webapps/wf_sec_storage/loader-security-properties.jar
cd /opt/workfusion/apps/webapps/wf_sec_storage/
/opt/workfusion/apps/webapps/wf_sec_storage/loader.sh workfusion review > ~/security-all.properties
    1. Copy the security-all.properties file from APP server to DB server using scp, ftp, or wget.

  5. Rename directory with the 8.4 installation. Run as root on APP server:

    mv /opt/workfusion /opt/workfusion_old

  6. Remove all Workfusion related cron jobs. Before deleting the jobs, make sure any non-Workfusion related cron jobs are backed up:

    crontab -l

    Run as root on APP server to delete all cronjobs:

    crontab -e

  7. Remove old init scripts (except for nginx as it will be needed for making backups of S3 buckets). Run as root on APP server:

    cd /etc/init.d
rm -f rpa_manager tabula tomcat tomcat-nlp wfagent php-fpm

PM Server

  1. Create dump. Run as wfuser on PM server:

    mkdir /opt/workfusion/dumps
/opt/workfusion/lib/mongodb/mongo --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" admin
mongo.shell> db.grantRolesToUser("admin",["backup"])
mongo.shell> quit()
/opt/workfusion/lib/mongodb/mongodump --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" --archive=/opt/workfusion/dumps/mongo-dump.tar

  2. Stop all old services. Run as wfuser on PM server

    mongod-service stop
wfagent-service stop

    Run as root on PM server:

    mv /opt/workfusion /opt/workfusion_old

  3. Clean up init services. Run as root on PM server:

    cd /etc/init.d
rm -f mongod wfagent

OCR Server

  1. Stop all old services. Run as root on OCR server:

    /etc/init.d/abbyy-fre11-licensing-service stop
/etc/init.d/wfagent stop
/etc/init.d/ocr2 stop

  2. Delete all old services. Run as root on OCR server:

    /etc/init.d/abbyy-fre11-licensing-service stop
/etc/init.d/wfagent stop
/etc/init.d/ocr2 stop
...

  3. Delete old ABBYY. Run as root on OCR server:

    cd /usr/local/bin
rm -rf ABBY
cd /usr/local/lib
rm -rf ABBY

  4. Remove all Workfusion related cron jobs. Before deleting the jobs, make sure any non-Workfusion related cron jobs are backed up:

    crontab -l

    Run as root on OCR server to delete all cronjobs:

    crontab -e

  5. Rename the directory with the 8.4 installation.

    Run as root on OCR server:

    mv /opt/workfusion /opt/workfusion_old

VDS Master Server

  1. Stop all old services. Run as wfuser on VDS Master server:

    wfmanager stop all

    Run as root on VDS Master server:

    service supervisord stop

  2. Copy the old models file from s3://vds-models/models.prod to a temporary directory. Run as wfuser on VDS Master server:

    s3cmd -c /opt/workfusion/s3cmd_conf/vds_s3cfg --signature-v2 sync s3://vds-models/models.flat-prod /tmp/models.prod-8.4

  3. Clear crontab jobs. Run as wfuser on VDS Master server to delete all cronjobs

    crontab -e

  4. Rename the directory with the 8.4 installation. Run as root on VDS Master server:

    mv /opt/workfusion /opt/workfusion_old

VDS Slave Server

  1. Stop all old services. Run as wfuser on VDS Slave servers:

    wfmanager stop all

    Run as root on VDS Slave servers:

    service supervisord stop

  2. Rename the directory with the 8.4 installation. Run as root on VDS Slave servers:

    mv /opt/workfusion /opt/workfusion_old

DB Server

  1. Connect to DB server via SSH.

  2. Find the environment.conf file used for the 8.4 installation.

  3. Convert usernames and secrets from environment.conf to YAML format used in the 9.0.x installation.

    Run as wfuser on DB server:

    cd /opt/workfusion/app-all/

    source ./environment.conf
cat << EOF > secrets_override.yml

workfusion_db_user: $WORKFUSION_MYSQL_USER
workfusion_db_pass: $WORKFUSION_MYSQL_PASS
postgresql_admin_pass: $POSTGRES_PASS

workfusion_datastore_user: $WORKFUSION_PSQL_USER
workfusion_datastore_pass: $WORKFUSION_PSQL_PASS

rpa_manager_db_admin_user: $MONGO_ADMIN_NAME
rpa_manager_db_admin_pass: $MONGO_ADMIN_PASS

wfagent_master_db_user: $MONGO_ADMIN_NAME # ??
wfagent_master_db_pass: $MONGO_ADMIN_PASS # ??

wfagent_mongodb_ro_user: $MONGO_ADMIN_RO_NAME
wfagent_mongodb_ro_pass: $MONGO_ADMIN_RO_PASS

wfagent_username: $WFAGENT_AUTH_USER
wfagent_password: $WFAGENT_AUTH_PASSWORD

ocr_passwd: $OCR_PASSWD

s3_access_key: $ACCESS_KEY
s3_secret_key: $SECRET_KEY

wfbi_db_user: $WFBI_USER
wfbi_db_pass: $WFBI_PASSWORD

apm_hostname: $APMHOSTNAME
db_hostname: $DBHOSTNAME
app_hostname: $WFHOSTNAME
ocr_hostname: $OCRHOSTNAME
vds_master_hostname: $MLHOSTNAME
EOF

  4. Copy the resulting secrets_override.yml file to the directory with the 9.0.x installer.

Data backup

Dumps directory

Create a directory where you store the dumps, for example, /opt/workfusion/dumps. Run as wfuser on DB server:

mkdir /opt/workfusion/dumps

MySQL

Run as wfuser on DB server:

sudo -u wfuser -i
/opt/workfusion/lib/mysql/bin/mysqldump --max_allowed_packet=1024M --add-drop-database --routines -S /opt/workfusion/apps/mysql/mysql.sock -u root -p -B sqc -B wfdb  >> /opt/workfusion/dumps/mysql-sqc_wfdb.sql

PostgreSQL

Run as wfuser on DB server:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/workfusion/lib/pgsql-9.4.7/lib
/opt/workfusion/lib/pgsql-client-9.4.7/bin/pg_dumpall -U postgres -h 127.0.0.1 -p 5432 > /opt/workfusion/dumps/postgres-all.sql

MongoDB

Run as wfuser on DB server:

/opt/workfusion/lib/mongodb/mongo --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" admin
mongo.shell> db.grantRolesToUser("admin",["backup"])
mongo.shell> quit()
/opt/workfusion/lib/mongodb/mongodump --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" --archive=/opt/workfusion/dumps/mongo-dump.tar

S3Emu

Run as wfuser on DB server:

s3emu-service stop
tar -czvf /opt/workfusion/dumps/riak.tar.gz /opt/workfusion/apps/s3emu/riak

Now, you can log in to APP server to stop nginx and remove the Nginx startup script from /etc/init.d:

Run on APP server:

# run as wfuser:
nginx-service stop

# run as root:
cd /etc/init.d
rm -f nginx

Solr

Run as wfuser on DB server:

solr-service stop
# the next command is optional since solr is no more used in default installation of SPA 9.x.
tar -czvf /opt/workfusion/dumps/solr.tar.gz /opt/workfusion/apps/solr

Nexus

Run as wfuser on DB server:

nexus stop
tar -czvf /opt/workfusion/dumps/nexus.tar.gz /opt/workfusion/apps/nexus/sonatype-work/nexus

Stop running services

Stop the remaining running services.

Run as wfuser on DB server:

mongod-service stop
mysql-service stop
postgresql-9.4-service stop
vault-service stop
wfagent-service stop

Make sure there are no running processes launched from wfuser.

Old installation renaming

Rename the old installation directory. Run as root on DB server:

mv /opt/workfusion /opt/workfusion_old

Installation and data restoring

DB Server

DB Server installation

Follow the installation for DB server. Apply secrets_override.yml with the -e @secrets_override.yml parameter.

./install.sh precheck db -e @secrets_override.yml
./install.sh preinstall db -e @secrets_override.yml
./install.sh install db -e @secrets_override.yml
./install.sh check db -e @secrets_override.yml

cd /opt/workfusion_old/dumps/

S3Emu

Replace s3emu data. Run as wfuser on DB server:

wfmanager stop riak riak-cs riak-stanchion
tar -xzvf riak.tar.gz
mv /opt/workfusion/riak/data/ /opt/workfusion/riak/data-bak
mkdir /opt/workfusion/riak/data
cp -r /opt/workfusion_old/dumps/opt/workfusion/apps/s3emu/riak/* /opt/workfusion/riak/data/

The extracted riak directory should containt two subfolders only: leveldb and bitcask.

MySQL

This need to be done only if new installation URL differs from previous. For example, if port changed, or URL switched from http to https.

In case your new environment runs on a port that differs from the one previously set, you should use the sed command in accordance with the new enviroment URL.

Example 1

The old environment uses the port 443 non-TLS while the new one works on the port 8443 with TLS.

So the sed command to replace the url is as follows:

sed -i "s/http\:\/\/old-wf\.workfusion\.com/https\:\/\/new-wf\.workfusion\.com\:8443/g" mysql-sqc_wfdb.sql

Example 2

The old environment uses the port 443 TLS while the new one should work on the port 8443 with TLS.

sed -i "s/https\:\/\/old-wf\.workfusion\.com\/https\:\/\/new-wf\.workfusion\.com\:8443/g" mysql-sqc_wfdb.sql

Upload the dump to MySQL

mysql -u mysql -h 127.0.0.1 -p < mysql-sqc_wfdb.sql

PostgreSQL

Connect to PostgreSQL database.

psql -U postgres -h 127.0.0.1

Run the next queries in the PostgreSQL shell to remove connections that may prevent dropping of databases (you can copy and paste all of them to the PSQL shell and press Enter):

SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'virtualizer_sandbox'
AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'virtualizer'
AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'wf_datastore'
AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'wfml_automation'
AND pid <> pg_backend_pid();

drop database virtualizer_sandbox;
drop database virtualizer;
drop database wf_datastore;
drop database wfml_automation;

This need to be done only if new installation URL differs from previous. For example if port changed, or URL switched from http to https.

In case your new environment runs on a port that differs from the one previously set, you should use the sed command in accordance with the new enviroment URL.

Example 1

The old environment uses the port 443 non-TLS while the new one works on the port 8443 with TLS.

So the sed command to replace the url is as follows:

sed -i "s/http\:\/\/old-wf\.workfusion\.com/https\:\/\/new-wf\.workfusion\.com\:8443/g" postgres-all.sql

Example 2

The old environment uses the port 443 TLS while the new one should work on the port 8443 with TLS.

sed -i "s/https\:\/\/old-wf\.workfusion\.com\/https\:\/\/new-wf\.workfusion\.com\:8443/g" postgres-all.sql

Restore data from the PostgreSQL 8.4 dump:

psql -U postgres -h 127.0.0.1 -f postgres-all.sql

MongoDB

Restore the database from backup:

mongo --host 127.0.0.1 --port 27016 --username admin --password "myadminpass" admin
mongo.shell> db.grantRolesToUser("admin",["root"])
mongorestore --host 127.0.0.1 --port 27016 --username admin --password "myadminpass" --archive=mongo-dump.tar

Nexus

tar -xvzf nexus.tar.gz
cd opt/workfusion/apps/nexus/
wfmanager stop nexus
cp -r sonatype-work/* /opt/workfusion/nexus/sonatype-work/

Solr

This step is optional. Since version 9.0.x Solr is not part of the default installation.

tar -xvzf solr.tar.gz
wfmanager stop solr
cp -r opt/workfusion/apps/solr/* /opt/workfusion/solr/

DB Migrations

Apply DB migrations:

cd /opt/workfusion/_migrations/workfusion
migrate pending --env=development
migrate up --env=development
cd /opt/workfusion/_migrations/workspace
migrate pending --env=development
migrate up --env=development
cd /opt/workfusion/_migrations/sandbox
migrate pending --env=development
migrate up --env=development
cd /opt/workfusion/_migrations/sqc
migrate pending --env=development
migrate up --env=development

Vault

Upload the user secrets from 8.4 Business Process to Vault:

cp ~/security-all.properties /opt/workfusion/wf-sec-storage
cd /opt/workfusion/wf-sec-storage
cat security-all.properties | grep -v 'This property is not configured.' |grep -v "It's all" | grep -v 'Key : Value' | grep -v 'You are working with safe' | grep -v 'Properties from secure storage' |sed s%\\[INFO\ \\]\ %%g | sed s/\ :\ /=/ > security-filtered-all.properties
./loader.sh workfusion security-filtered-all.properties

Final migrations

To apply final migrations, go to the directory with the ansible 9.0.x installer. Run as wfuser.

./install.sh install db -e @secrets_override.yml
./install.sh check db -e @secrets_override.yml

APP Server

  1. Go to the directory with the ansible 9.0.x installer.

    cd <directory_with_installer_9.0.x>
cp /opt/workfusion_old/apps/webapps/apache-tomcat-8.5.20/conf/license.properties .

  2. Follow the installation for APP server. Apply secrets_override.yml with the -e @secrets_override.yml parameter.

    • Run as root.
    ./install.sh precheck app -e @secrets_override.yml
./install.sh preinstall app -e @secrets_override.yml
    • Run as wfuser
    ./install.sh install app -e @secrets_override.yml
./install.sh check app -e @secrets_override.yml

  3. Copy RPA Manager quota to the new installation.

    cp /opt/workfusion_old/apps/rpa_manager/grid-router/quota/user.xml /opt/workfusion/rpa-manager/grid-router/quota/user.xml

  4. Re-log in as wfuser otherwise the wfmanager executable will not be found.

    wfmanager restart all

PM Server

  1. Go to the directory with the ansible 9.0.x installer.

    cd <directory_with_installer_9.0.x>

  2. Follow the installation for PM server. Apply secrets_override.yml with the -e @secrets_override.yml parameter.

    • Run as root.
    ./install.sh precheck apm -e @secrets_override.yml
./install.sh preinstall apm -e @secrets_override.yml
    • Run as wfuser.
    ./install.sh install apm -e @secrets_override.yml
./install.sh check apm -e @secrets_override.yml

  3. Re-log in as wfuser otherwise the wfmanager executable will not be found.

    mongorestore --host 127.0.0.1 --port 13017 --username mongodb --password "mongodb" --archive=/opt/workfusion_old/dumps/mongo-dump.tar

If you have any custom APM configuration like checks, transfer them to new APM server.

OCR Server

Installation

  1. Go to the directory with the ansible 9.0.x installer.

    cd <directory_with_installer_9.0.x>

  2. Follow the installation for OCR server. Apply secrets_override.yml with the -e @secrets_override.yml parameter

    • Run as root.
    ./install.sh precheck ocr -e @secrets_override.yml
./install.sh preinstall ocr -e @secrets_override.yml
    • Run as wfuser
    ./install.sh install ocr -e @secrets_override.yml -e @secrets_override.yml
./install.sh check ocr -e @secrets_override.yml -e @secrets_override.yml

License

note

Actions below should be repeated for all activated licenses for all OCR servers.

  1. Run License Manager using any user.

    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<INSTALL_DIR>/apps/ABBYY/FREngine11/Bin
<INSTALL_DIR>/apps/ABBYY/FREngine11/Bin/LicenseManager.Console

  2. License serial number and other license parameters can differ from samples. Select the activated license:

  3. Open license parameters:

  4. Make a screenshot of the first page. Ensure that line with Volume > Regular Text > Remains is visible.

  5. Request and activate license using the Activating OCR License guide. Attach all license screenshots if remaining is greater than 0.

API

Pattern usage

Changes below are required in case custom pattern is used in BP.

See full API reference.

  1. replace patternName parameter with pattern parameter that is passed by content, not file name.

    Affected endpoints:

    • /processImage
    • /submitPattern - added
    • /trainPattern - renamed to mergePattern that is combined with new one
    • /processDocument - uses pattern submitted with /submitPattern

  2. Use useDefaultPattern=true instead of patternName=default

    Affected endpoints:

    • /processImage
    • /processDocument

OCR parameters

Collect all changed parameters of OCR service in YML files (the /etc directory) comparing with standard listed in blocks below:

ocr-rest.yml

standard OCR rest properties

db.cleanup.abbyy:
 input: 2000
 output: 2000
 record: 600

spring.profiles.active: disabled-auth,mongoDbQueue,gridfs-storage,mongo-task 

ocr.rest.timeout: 5
ocr.rest.retries: 2

ocr-worker.yml

standard OCR worker properties

worker.executor.abbyy.pool.size: 

spring.profiles.active: abbyy,mongoDbQueue,gridfs-storage

Update configuration YML files with collected parameter values on the previous step.

Files to review in case any customization exists:

  • ocr-rest.yml
  • ocr-worker.yml

OCR GhostScript

A new version of GhostScript is available since 8.5.x SPA version.

Newer version of GhostScript has impact on document convertion to tiff and may change quality of OCR.

In case one of pre-processing steps uses GhostScript before OCR and quality is worse, there's need to downgrade version of GhostScript in Bot Task (PDF to Tiff convert) of Business Process:

"gs" is a new version, "gs919" is an old version.

VDS Master

  1. Connect to VDS Master Server.

  2. Go to the directory with the ansible 9.0.x installer.

    cd <directory_with_installer_9.0.x>

  3. Follow the installation for VDS Master. Apply secrets_override.yml with the -e @secrets_override.yml parameter.

    • Run as wfuser:
    ./install.sh precheck vds-master -e @secrets_override.yml
./install.sh preinstall vds-master -e @secrets_override.yml
./install.sh install vds-master -e @secrets_override.yml
./install.sh check vds-master -e @secrets_override.yml

  4. Copy the new models file from s3://vds-models/models.flat-prod to a temporary directory:

    s3cmd -c /opt/workfusion/s3cmd_conf/vds_s3cfg --signature-v2 sync s3://vds-models/models.flat-prod /tmp/
s3cmd -c /opt/workfusion/s3cmd_conf/vds_s3cfg --signature-v2 sync s3://vds-models/.models.list /tmp/

  5. Merge these files to a new one:

    cat /tmp/models.prod-8.4 >> /tmp/models.flat-prod
cat /tmp/models.prod-8.4 >> /tmp/.models.list

  6. Upload the merged file to s3://vds-models/models.flat-prod:

    s3cmd -c /opt/workfusion/s3cmd_conf/vds_s3cfg --signature-v2 sync /tmp/models.flat-prod s3://vds-models/models.flat-prod
s3cmd -c /opt/workfusion/s3cmd_conf/vds_s3cfg --signature-v2 sync /tmp/.models.list s3://vds-models/.models.list

  7. Restart wfmanager:

    wfmanager restart all

VDS Slave

  1. Go to the directory with the ansible 9.0.x installer.

    cd <directory_with_installer_9.0.x>

  2. Follow the installation for VDS Slave. Apply secrets_override.yml with the -e @secrets_override.yml parameter

    • Run as root.
    ./install.sh precheck vds-slave -e @secrets_override.yml
./install.sh preinstall vds-slave -e @secrets_override.yml
    • Run as wfuser.
    ./install.sh install vds-slave -e @secrets_override.yml
./install.sh check vds-slave -e @secrets_override.yml

Post-installation steps

Perform post-installation procedures as described in Post installation.

← PreviousNext →