9.0

Edit

Upgrade from 8.2.3 to 9.0.x

Prerequisites

WorkFusion SPA 8.2.X is operating on RHEL 6, while 8.5.X and higher require RHEL 7.3 or RHEL 7.4. So in-place upgrade is not possible.

WorkFusion 8.5.x should be installed prior to migration.

Provisions

The following provisions should be observed when using this guide.

WorkFusion Environments

  • old-wf.workfusion.com: APP Server name in 8.2.3, for example, onprem-v2-552-8-2-3-wf.workfusion.com
  • new-wf.workfusion.com: WorkFusion Server names in 9.0, for example, aafs-844-852-db1.wfwf-10005.workfusion.com

WorkFusion installation directory: /opt/workfusion/

If in your environment WorkFusion is installed in another directory, use that directory instead.

WorkFusion default user

Default WorkFusion user is assumed as wfuser.

If in your environment the default user is different, use that user name instead.

Ports

Ports used in this guide are default. Consider to change them in case the different ports are configured in your environment.

Preparation

To successfully migrate all the platform components to a new environment, backup all the data in the storages and the databases.

  1. Create a directory for dumps:

    mkdir /opt/workfusion/dumps

  2. Stop the following services to proceed with the backup and make sure all the data are consistent.

    /opt/workfusion/apps/wfagent/wfagent stop
/opt/workfusion/apps/nexus/nexus/bin/nexus stop
/opt/workfusion/apps/s3emu/riak.service stop

Data backup

MySQL

sudo -u wfuser -i
/opt/workfusion/lib/mysql/bin/mysqldump --max_allowed_packet=1024M --routines -S /opt/workfusion/apps/mysql/mysql.sock -u root -p -B sqc -B wfdb  >> /opt/workfusion/dumps/mysql-sqc_wfdb.sql

PostgreSQL

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/workfusion/lib/pgsql-9.4.7/lib
/opt/workfusion/lib/pgsql-client-9.4.7/bin/pg_dumpall -U postgres -h 127.0.0.1 -p 5432 > /opt/workfusion/dumps/postgres-all.sql

MongoDB

/opt/workfusion/lib/mongodb/mongodump --host 127.0.0.1 --port 27017 --username admin --password "myadminpass" --archive=/opt/workfusion/dumps/mongo-dump.tar

In case you going to change S3 object storage to Minio, use section S3EMU Minio.

S3Emu

Find and save the riak-cs AWS keys from the 8.2.3 installation. They are stored as a plain text in the /opt/workfusion/apps/s3emu/etc/s3cfgfasttrack file on APP Server.

s3emu-service stop
tar -czvf /opt/workfusion/dumps/riak.tar.gz /opt/workfusion/apps/s3emu/riak

S3Emu Minio

  1. Check how much space your S3 storage takes:

    s3cmd -c /opt/workfusion/apps/s3emu/etc/s3cfgfasttrack du -H | grep Total

    s3cmd -c /opt/workfusion/apps/s3emu/etc/s3cfgfasttrack du -H | grep Total

    Ensure that you have enough disk space.

  2. Create a script that backs up S3 data:

    cat << 'EOF' > backup-s3.sh
#!/bin/bash
BUCKET_LIST=$(s3cmd --signature-v2 -c /opt/workfusion/apps/s3emu/etc/s3cfgfasttrack  ls | cut -d'/' -f3); \
for i in $BUCKET_LIST; do mkdir -p /opt/workfusion/dumps/wf-s3-content/$i && \
s3cmd --signature-v2 -c /opt/workfusion/apps/s3emu/etc/s3cfgfasttrack get -f -r s3://$i /opt/workfusion/dumps/wf-s3-content/$i; done
EOF

  3. Make it executable and launch it:

    chmod +x backup-s3.sh
./backup-s3.sh

  4. Archive wf-s3-content:

    tar -czvf /opt/workfusion/dumps/wf-s3-content.tar.gz /opt/workfusion/dumps/wf-s3-content

Solr

solr-service stop
tar -czvf /opt/workfusion/dumps/solr.tar.gz /opt/workfusion/apps/solr

Nexus

nexus stop
tar -czvf /opt/workfusion/dumps/nexus.tar.gz /opt/workfusion/apps/nexus/sonatype-work/nexus

Check the additional data and passwords on the Application server and save them in the dumps directory:

cp -pr /opt/workfusion/webapps/apache-tomcat-8.0.33/conf/secure* /opt/workfusion/dumps

To speed up the download, all data can be zipped.

tar -czvf /opt/workfusion/dump.tar.gz /opt/workfusion/dumps

Data migration

  1. Copy dumps to the new DB Server. For this purpose, you can use any application which is the best for you.

    The default directory for dumps: /opt/dump.

  2. Populate secrets.yml with the AWS access and secret keys from 8.2.3 installation. If you going to use Minio as S3 Object Storage use AWS access keys from 9.0.x instantiation.

    Example keys

    s3_access_key:       JOZXPHL1ZC8WXZAJ1111
s3_secret_key:       qAEYOVLOViPjxk3Lbuj0wrDztf8fGJrqM1111w==

  3. Unpack the archive with dumps.

  4. Move to folder where dumps were stored on DB server.

    tar zxvf dump.tar.gz

  5. Go to the directory where you have extracted the dumps to.

Data restoring

S3Emu Minio

Unpack the archive with S3Emu backup from 8.2.3.

tar -xzvf wf-s3-content.tar.gz
cp -r opt/workfusion/dumps/wf-s3-content/* /opt/workfusion/minio/data/

S3Emu

Unpack the archive with S3Emu backup from 8.2.3.

tar -xzvf riak.tar.gz
cp -r /opt/dump/dump/riak/* /opt/workfusion/riak/data/

The extracted riak directory should containt two subfolders only: leveldb and bitcask.

MySQL

mysql -u mysql -p -h 127.0.0.1

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sqc                |
| sys                |
| wfdb               |
+--------------------+
6 rows in set (0.00 sec)

mysql> drop database sqc;
Query OK, 5 rows affected (0.03 sec)

mysql> drop database wfdb;
Query OK, 202 rows affected (1.04 sec)

mysql> quit

Replace the old URL and the user with the new ones in the MySQL dump.

important

This need to be done only if new installation URL differs from previous. For example, if the port changed, or URL switched from HTTP to HTTPS.

In case your new environment runs on a port that differs from the one previously set, you should use the sed command in accordance with the new enviroment URL.

Example 1

The old environment uses the port 80 non-TLS while the new one works on the port 8443 with TLS.

So the sed command to replace the url is as follows:

sed -i "s/http\:\/\/old-wf\.workfusion\.com/https\:\/\/new-wf\.workfusion\.com\:8443/g" mysql-dump.sql

Example 2

The old environment uses the port 443 TLS while the new one should work on the port 8443 with TLS.

sed -i "s/https\:\/\/old-wf\.workfusion\.com\:443/https\:\/\/new-wf\.workfusion\.com\:8443/g" mysql-dump.sql
or
sed -i "s/https\:\/\/old-wf\.workfusion\.com/https\:\/\/new-wf\.workfusion\.com\:8443/g" mysql-dump.sql

  1. Replace definer user for stored procedures. In the code below, WORKFUSION_DB_USER is the user from secrets.yml.

    sed -i s/mysqluser/WORKFUSION_DB_USER/g mysql-dump.sql

  2. Upload the dump to MySQL.

    mysql -u mysql -h 127.0.0.1 -p < mysql-sqc_wfdb.sql

PostgreSQL

  1. Connect to PostgreSQL database:

    psql -U postgres -h 127.0.0.1

  2. Run the next queries in the PostgreSQL shell to remove connections that may prevent dropping of databases (you can copy and paste all of them to thePSQLshell and press Enter):

    SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'virtualizer_sandbox'
  AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'virtualizer'
  AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'wf_datastore'
  AND pid <> pg_backend_pid();
SELECT pg_terminate_backend(pg_stat_activity.pid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'wfml_automation'
  AND pid <> pg_backend_pid();

drop database virtualizer_sandbox;
drop database virtualizer;
drop database wf_datastore;
drop database wfml_automation;

  3. Replace the old URL in PostgreSQL 8.2.3 dump with the new one.

important

In case your new environment runs on a port that differs from the one previously set, you should adjust the sed command in accordance with the new enviroment URL.

Example

The old environment uses the port 4443 while the new one works on the port 8443.

So the sed command to replace the url is as follows:

sed -i "s/old-wf\.workfusion\.com\:4443/new-wf\.workfusion\.com\:8443/g" postgres-all.sql

sed -i "s/old-wf\.workfusion\.com/new-wf\.workfusion\.com/g" postgres-all.sql

Restore data from the PostgreSQL 8.2.3 dump:

psql -U postgres -h 127.0.0.1 -f postgres-all.sql

In the code above, postgres is the PostrgeSQL user from secrets.yml.

MongoDB

Restore the database from backup

mongorestore --host 127.0.0.1 --port 27016 --username mongodb --password "mongodb" --archive=mongo-dump.tar

Nexus

tar -xvzf nexus.tar.gz
cd opt/workfusion/apps/nexus/
wfmanager stop nexus
cp -r sonatype-work/* /opt/workfusion/nexus/sonatype-work/

Solr

wfmanager stop solr
cp -r solr/* /opt/workfusion/solr/

DB

  1. Apply DB migrations:

    cd /opt/workfusion/_migrations/workfusion
migrate pending --env=development
migrate up --env=development
cd /opt/workfusion/_migrations/workspace
migrate pending --env=development
migrate up --env=development
cd /opt/workfusion/_migrations/sandbox
migrate pending --env=development
migrate up --env=development
cd /opt/workfusion/_migrations/sqc
migrate pending --env=development
migrate up --env=development

  2. Go to the installer directory.

  3. Run as wfuser

    ./install.sh install db

Vault

  1. Upload the user secrets from 8.2.3 Business Process to Vault:

    cd /opt/workfusion/wf-sec-storage
mkdir _upgrade
cp secure-store*.txt _upgrade/

  2. Use the following command to list all secrets:

    cd wf-sec-storage
./loader.sh workfusion show

  3. Copy the output to a text file (prop.file).

  4. In the text file substuitute : with = and remove the spaces before and after =.

  5. Run the following command to update the secure properties:

    ./loader.sh workfusion prop.file

  6. Restart tomcat.

APP Server upgrade

  1. Connect to APP Server.

  2. Apply migrations for SQC, Virtualizer, Virtualizer-sanbdox databases.

    cd /opt/workfusion/workfusion/migrations_workfusion
migrate pending --env=development
migrate up --env=development

  3. Go to the installer directory.

  4. Run as wfuser.

    ./install.sh install app

  5. Restart applications to apply changes.

    wfmanager restart all

VDS Server upgrade

  1. Connect to VDS Master Server.

  2. Go to the installer directory.

  3. Run as wfuser.

    ./install.sh install vds-master

  4. Copy the old models file from s3://vds-models/models.prod to a temporary directory:

    s3cmd -c /opt/workfusion/s3cmd_conf/workfusion_s3cfg --signature-v2 sync s3://vds-models/models.prod /tmp/

  5. Copy the new models file from s3://vds-models/models.flat-prod to a temprorary directory:

    s3cmd -c /opt/workfusion/s3cmd_conf/workfusion_s3cfg --signature-v2 sync s3://vds-models/models.flat-prod /tmp/

  6. Merge these files to a new one:

    cat /tmp/models.prod >> /tmp/models.flat-prod

  7. Upload the merged file to s3://vds-models/models.flat-prod.

    s3cmd -c /opt/workfusion/s3cmd_conf/workfusion_s3cfg --signature-v2 sync /tmp/models.flat-prod s3://vds-models/models.flat-prod

RPA Server upgrade

Perform the upgrade steps for RPA Server as described in RPA Server upgrade.

APM Server upgrade

If you have any custom APM configuration like checks, transfer them to a new APM server.

OCR server upgrade

License

note

Actions below should be repeated for all activated licenses for all OCR servers.

  1. Run License Manager using any user.

    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<INSTALL_DIR>/apps/ABBYY/FREngine11/Bin
<INSTALL_DIR>/apps/ABBYY/FREngine11/Bin/LicenseManager.Console

    License serial number and other license parameters can differ from samples.

  2. Select the activated license:

  3. Open license parameters:

  4. Make a screenshot of the first page. Ensure that line with Volume > Regular Text > Remains is visible.

  5. Request and activate license using the Activating OCR License guide. Attach all license screenshots if remaining is greater than 0.

API

Pattern usage

Changes below are required in case custom pattern is used in BP.

See the full OCR API reference.

Replace the patternName parameter with the pattern parameter that is passed by content, not filename.

Affected endpoints:

  • /processImage
  • /submitPattern: added
  • /trainPattern: renamed to mergePattern that is combined with a new one
  • /processDocument: uses pattern submitted with /submitPattern

Use useDefaultPattern=true instead of patternName=default.

Affected endpoints:

  • /processImage
  • /processDocument

OCR parameters

Collect all changed parameters of OCR service in YML files (the /etc directory) comparing with standard listed in blocks below:

ocr-rest.yml

standard OCR rest properties

db.cleanup.abbyy:
 input: 2000
 output: 2000
 record: 600

spring.profiles.active: disabled-auth,mongoDbQueue,gridfs-storage,mongo-task 

ocr.rest.timeout: 5
ocr.rest.retries: 2

ocr-worker.yml

standard OCR worker properties

worker.executor.abbyy.pool.size: 

spring.profiles.active: abbyy,mongoDbQueue,gridfs-storage

Update configuration YML files with collected parameter values on the previous step.

Files to review in case any customization exists:

  • ocr-rest.yml
  • ocr-worker.yml

OCR GhostScript

A new version of GhostScript is available since 8.5.x SPA version.

Newer version of GhostScript has impact on document convertion to tiff and may change quality of OCR.

In case one of pre-processing steps uses GhostScript before OCR and quality is worse, there's need to downgrade version of GhostScript in Bot Task (PDF to Tiff convert) of Business Process:

"gs" is a new version, "gs919" is an old version.

← PreviousNext →