9.4

Edit

MinIO upgrade and migration

The guide covers the Ansible-based Minio upgrade. If you wish to perform manual Minio installation, refer to use Migration from Riak to Minio.

The guide covers Ansible-based Minio upgrade from Riak to Minio. Since 9.0.4 Minio is using as default s3 backend.

Prerequisites

  • 9.1 installer or higher is transferred to db server.

Provisions

The following provisions should be observed when using this guide.

  • WorkFusion Installer Directory: the directory where the installer is located; referred as <PACKAGE_DIR>.

  • WorkFusion Home Directory: referred as <INSTALL_DIR>. We assume, that WorkFusion Home Directory is /opt/workfusion/.

    note

    If WorkFusion is installed in another directory in your environment, use that directory instead.

  • WorkFusion default user: the default WorkFusion user is assumed to be wfuser.

    note

    If default user differs in your environment, use a respective username instead.

Upgrade steps

Navigate to WorkFusion Installer Directory:

cd <PACKAGE_DIR>

DB server

To perform upgrade to Minio:

  1. Edit the wf_s3_backend variable from riak to minio in config_overrides.yml. Use any available editor ("vi" is used in the example below):

    vi config_overrides.yml

  2. Perform precheck and preinstall operations. Run as root:

    ./install.sh precheck <SERVER_ROLE> -e @config_overrides.yml -e @secrets_overrides.yml
./install.sh preinstall <SERVER_ROLE> -e @config_overrides.yml -e @secrets_overrides.yml

  3. Export the WorkFusion user. Execute this step if the WorkFusion user is not wfuser.

    Run as wfuser:

    export WFUSER=<value of 'wf_user' from 'config_overrides.yml'>

  4. Perform installation and post-installataion check operations. Run as wfuser:

    ./install.sh install <SERVER_ROLE> -e @config_overrides.yml -e @secrets_overrides.yml
./install.sh check <SERVER_ROLE> -e @config_overrides.yml -e @secrets_overrides.yml

Check Minio availability

  1. Ensure that DNS names for servers are configured according to the DNS names configuration section.

  2. Check that Minio Client (mc) exists in <INSTALL_DIR>/tool directory:

    1. SSH into the <server_role> server.

    2. Run as wfuser:

    cd <INSTALL_DIR>/tools
./mc -C <INSTALL_DIR>/minio_client_conf/ config host list

    The similar output should appear:

    gcs  :  https://storage.googleapis.com  YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  S3v2   dns  
local:  http://localhost:9000                                                              auto 
minio:  http://localhost:9000           YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  s3v4   auto
play :  https://play.minio.io:9000      YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  S3v4   auto 
riak :  http://localhost:8081           YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  s3v2   auto 
s3   :  https://db-server-host-name     YOUR-ACCESS-KEY-HERE  YOUR-SECRET-KEY-HERE  s3v2   auto

  3. Add Riak to mc hosts:

    cd <INSTALL_DIR>/tools
./mc -C <INSTALL_DIR>/minio_client_conf/ config host add riak http://127.0.0.1:8081 <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY> --api s3v2 --insecure

  4. Check that Riak buckets are not empty:

    cd <INSTALL_DIR>/tools
./mc -C <INSTALL_DIR>/minio_client_conf/ ls riak/doc-upload

    The similar output should appear:

    [2018-08-17 11:07:05 UTC]  17KiB 08b66fcb-8e9c-4931-a177-f9002157b9ba.pdf
[2018-08-17 11:08:00 UTC]   512B 12bb166d-a847-42a0-a49f-04e0de500da6.html
[2018-08-17 11:08:40 UTC]   512B 7afd9205-04a6-423b-88ef-aae30b8b4d81.html
[2018-08-17 11:07:05 UTC]   512B 9615fba7-62ab-4250-a22d-03a50d6315f2.html
[2018-08-17 11:06:18 UTC]  17KiB a0ae9da5-2ce7-4511-887d-a0fa87a27b1f.pdf
[2018-08-17 11:06:19 UTC]   512B ba400afb-1045-4969-b3da-81b50876ce9d.html
[2018-08-17 11:08:40 UTC]  17KiB db299e4b-f081-419e-a62b-0ae50c9c9f86.pdf
[2018-08-17 11:08:00 UTC]  17KiB f36f2b21-277e-41cb-b45c-ce241578e02b.pdf

Migration steps

  1. Check the already used disk space by Riak to ensure that <INSTALL_DIR> has enough capacity to create the same amount of data for Minio migration (2x size).

  2. List all buckets:

    s3cmd --signature-v2 -c /opt/workfusion/s3cmd_conf/workfusion_s3cfg ls

2018-08-20 11:31 s3://doc-upload
2018-08-20 11:31 s3://recorder-images
2018-08-20 11:31 s3://secdocs
2018-08-20 11:31 s3://tinymce-resources

  3. Check buckets' size and sum output values:

    s3cmd --signature-v2 -c /opt/workfusion/s3cmd_conf/workfusion_s3cfg du s3://doc-upload
73400    8 objects s3://doc-upload/

To perform files migration from riak to minio:

  1. Execute the following command:
cd <INSTALL_DIR>/tools
./mc -C <INSTALL_DIR>/minio_client_conf/ mirror riak minio --overwrite

  1. Check if migration is successful:

    cd <INSTALL_DIR>/tools
./mc -C <INSTALL_DIR>/minio_client_conf/ ls minio/doc-upload

    The similar output should appear:

    [2018-08-17 11:07:05 UTC]  17KiB 08b66fcb-8e9c-4931-a177-f9002157b9ba.pdf
[2018-08-17 11:08:00 UTC]   512B 12bb166d-a847-42a0-a49f-04e0de500da6.html
[2018-08-17 11:08:40 UTC]   512B 7afd9205-04a6-423b-88ef-aae30b8b4d81.html
[2018-08-17 11:07:05 UTC]   512B 9615fba7-62ab-4250-a22d-03a50d6315f2.html
[2018-08-17 11:06:18 UTC]  17KiB a0ae9da5-2ce7-4511-887d-a0fa87a27b1f.pdf
[2018-08-17 11:06:19 UTC]   512B ba400afb-1045-4969-b3da-81b50876ce9d.html
[2018-08-17 11:08:40 UTC]  17KiB db299e4b-f081-419e-a62b-0ae50c9c9f86.pdf
[2018-08-17 11:08:00 UTC]  17KiB f36f2b21-277e-41cb-b45c-ce241578e02b.pdf

Riak cleanup

  1. Clear Riak buckets' data. Execute the following command:

    cd <INSTALL_DIR>/tools
./mc -C <INSTALL_DIR>/minio_client_conf/ rm --recursive --force --dangerous riak

    The similar output should appear:

    Removing `riak/doc-upload/260a5e68-617e-4720-a7cd-56183cd88223.pdf`.
Removing `riak/doc-upload/296e3683-f63e-4b25-81f4-484cee204e57.pdf`.
Removing `riak/doc-upload/3a74fbaa-f9b5-41f4-b35a-824da55f8e50.html`.
Removing `riak/doc-upload/46166147-3d95-4e6c-b73d-e3e2d3d3b071.html`.
Removing `riak/doc-upload/b7322349-5988-454e-b2f0-767b7759f5e5.html`.
Removing `riak/doc-upload/bf48afff-2578-4bc4-881d-e8c1ce43d303.pdf`.
Removing `riak/doc-upload/e009b010-898f-42a8-a97d-659c43fb1d42.html`.
Removing `riak/doc-upload/f34ee6eb-9aba-466c-8c84-4bec6ee4d36d.pdf`.
Removing `riak/doc-upload`.
Removing `riak/recorder-images`.
Removing `riak/secdocs`.
Removing `riak/tinymce-resources`.

  2. Stop all Riak components.

    cd <INSTALL_DIR>/
wfmanager stop riak-stanchion
wfmanager stop riak
wfmanager stop riak-cs

  3. Delete init scripts for Riak.

    cd <INSTALL_DIR>/supervisord/apps
rm riak.ini
rm riak-cs.ini
rm riak-stanchion.ini

  4. Delete Riak folders.

    cd <INSTALL_DIR>/
rm -rf riak
rm -rf riak-cs
rm -rf riak-stanchion
← PreviousNext →