Edit

MinIO upgrade and migration

This guide is covering ansible-based MinIO upgrade. If you wish to perform manual MinIO installation, see Migration from Riak to MinIO.

Since 9.0.4 Minio is used as default s3 backend.

Prerequisites

• 9.1 installer or higher is transferred to db server.

Provisions

The following provisions should be observed when using this guide.

WorkFusion Installer Directory

In this guide the directory where the installer is located is referred as<PACKAGE_DIR>.

WorkFusion Home Directory

In this guide WorkFusion Home Directory is referred as <INSTALL_DIR>.

We assume, that WorkFusion Home Directory is /opt/workfusion/

If WorkFusion is installed in another directory in your environment, please use that directory instead.

WorkFusion default user

Default WorkFusion user is assumed to be wfuser.

If default user differs in your environment, please use respective user name instead.

Upgrade steps

Navigate to WorkFusion Installer Directory first:

cd <PACKAGE_DIR>

DB server

1. To upgrade to MinIO, edit the wf_s3_backend variable from riak to minio in config_overrides.yml:

   Edit the following file with any available editor (vi is used in the example below):

   vi config_overrides.yml
   

2. Perform Precheck and Preinstall operations.

   Run as root:

   ./install.sh precheck <SERVER_ROLE> -e @config_overrides.yml -e @secrets_overrides.yml
   ./install.sh preinstall <SERVER_ROLE> -e @config_overrides.yml -e @secrets_overrides.yml
   

3. Export a WorkFusion user.

   Execute this step when WorkFusion user is not wfuser

   Run as wfuser:

   export WFUSER=<value of 'wf_user' from 'config_overrides.yml'>
   

4. Perform Installation and Post-install Check operations.

   Run as wfuser:

   ./install.sh install <SERVER_ROLE> -e @config_overrides.yml -e @secrets_overrides.yml
   ./install.sh check <SERVER_ROLE> -e @config_overrides.yml -e @secrets_overrides.yml
   

Check MinIO availability

1. Ensure, that DNS names for servers are configured according to the DNS names configuration section.

   Check that MinIO Client (mc) exists in <INSTALL_DIR>/tool directory

   • SSH into <server_role> server.

   • Run as wfuser:

   cd <INSTALL_DIR>/tools
   ./mc -C <INSTALL_DIR>/minio_client_conf/ config host list
   

   The similar output should appear:

   gcs  :  https://storage.googleapis.com  YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  S3v2   dns  
   local:  http://localhost:9000                                                              auto 
   minio:  http://localhost:9000           YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  s3v4   auto
   play :  https://play.minio.io:9000      YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  S3v4   auto 
   riak :  http://localhost:8081           YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  s3v2   auto 
   s3   :  https://db-server-host-name     YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  s3v2   auto 
   

   Add Riak to mc hosts:

   cd <INSTALL_DIR>/tools
   ./mc -C <INSTALL_DIR>/minio_client_conf/ config host add riak http://127.0.0.1:8081 <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY> --api s3v2 --insecure
   

2. Check that Riak buckets are't empty:

   cd <INSTALL_DIR>/tools
   ./mc -C <INSTALL_DIR>/minio_client_conf/ ls riak/doc-upload
   

   The similar output should appear:

   [2018-08-17 11:07:05 UTC]  17KiB 08b66fcb-8e9c-4931-a177-f9002157b9ba.pdf
   [2018-08-17 11:08:00 UTC]   512B 12bb166d-a847-42a0-a49f-04e0de500da6.html
   [2018-08-17 11:08:40 UTC]   512B 7afd9205-04a6-423b-88ef-aae30b8b4d81.html
   [2018-08-17 11:07:05 UTC]   512B 9615fba7-62ab-4250-a22d-03a50d6315f2.html
   [2018-08-17 11:06:18 UTC]  17KiB a0ae9da5-2ce7-4511-887d-a0fa87a27b1f.pdf
   [2018-08-17 11:06:19 UTC]   512B ba400afb-1045-4969-b3da-81b50876ce9d.html
   [2018-08-17 11:08:40 UTC]  17KiB db299e4b-f081-419e-a62b-0ae50c9c9f86.pdf
   [2018-08-17 11:08:00 UTC]  17KiB f36f2b21-277e-41cb-b45c-ce241578e02b.pdf
   

Migration steps

Check already used disk space by Riak, to ensure that <INSTALL_DIR> has enough capacity to create the same amount of data for MinIO migration (2x size).

List all buckets:

s3cmd --signature-v2 -c /opt/workfusion/s3cmd_conf/workfusion_s3cfg ls

2018-08-20 11:31 s3://doc-upload
2018-08-20 11:31 s3://recorder-images
2018-08-20 11:31 s3://secdocs
2018-08-20 11:31 s3://tinymce-resources

Check buckets size, and sum output values:

s3cmd --signature-v2 -c /opt/workfusion/s3cmd_conf/workfusion_s3cfg du s3://doc-upload
73400    8 objects s3://doc-upload/

To perform files migration from Riak to MinIO:

1. Execute the following command bellow:

   cd <INSTALL_DIR>/tools
   ./mc -C <INSTALL_DIR>/minio_client_conf/ mirror riak minio --overwrite
   

2. Check if migration is successful:

   cd <INSTALL_DIR>/tools
   ./mc -C <INSTALL_DIR>/minio_client_conf/ ls minio/doc-upload
   

   The similar output should appear:

   [2018-08-17 11:07:05 UTC]  17KiB 08b66fcb-8e9c-4931-a177-f9002157b9ba.pdf
   [2018-08-17 11:08:00 UTC]   512B 12bb166d-a847-42a0-a49f-04e0de500da6.html
   [2018-08-17 11:08:40 UTC]   512B 7afd9205-04a6-423b-88ef-aae30b8b4d81.html
   [2018-08-17 11:07:05 UTC]   512B 9615fba7-62ab-4250-a22d-03a50d6315f2.html
   [2018-08-17 11:06:18 UTC]  17KiB a0ae9da5-2ce7-4511-887d-a0fa87a27b1f.pdf
   [2018-08-17 11:06:19 UTC]   512B ba400afb-1045-4969-b3da-81b50876ce9d.html
   [2018-08-17 11:08:40 UTC]  17KiB db299e4b-f081-419e-a62b-0ae50c9c9f86.pdf
   [2018-08-17 11:08:00 UTC]  17KiB f36f2b21-277e-41cb-b45c-ce241578e02b.pdf
   

Riak cleanup

Clear Riak buckets data.

Execute the following command bellow:

cd <INSTALL_DIR>/tools
./mc -C <INSTALL_DIR>/minio_client_conf/ rm --recursive --force --dangerous riak

The similar output should appear:

Removing `riak/doc-upload/260a5e68-617e-4720-a7cd-56183cd88223.pdf`.
Removing `riak/doc-upload/296e3683-f63e-4b25-81f4-484cee204e57.pdf`.
Removing `riak/doc-upload/3a74fbaa-f9b5-41f4-b35a-824da55f8e50.html`.
Removing `riak/doc-upload/46166147-3d95-4e6c-b73d-e3e2d3d3b071.html`.
Removing `riak/doc-upload/b7322349-5988-454e-b2f0-767b7759f5e5.html`.
Removing `riak/doc-upload/bf48afff-2578-4bc4-881d-e8c1ce43d303.pdf`.
Removing `riak/doc-upload/e009b010-898f-42a8-a97d-659c43fb1d42.html`.
Removing `riak/doc-upload/f34ee6eb-9aba-466c-8c84-4bec6ee4d36d.pdf`.
Removing `riak/doc-upload`.
Removing `riak/recorder-images`.
Removing `riak/secdocs`.
Removing `riak/tinymce-resources`.

Stop all Riak components.

cd <INSTALL_DIR>/
wfmanager stop riak-stanchion
wfmanager stop riak
wfmanager stop riak-cs

Delete init scripts for Riak.

cd <INSTALL_DIR>/supervisord/apps
rm riak.ini
rm riak-cs.ini
rm riak-stanchion.ini

Delete Riak folders.

cd <INSTALL_DIR>/
rm -rf riak
rm -rf riak-cs
rm -rf riak-stanchion