Log rotation

This article explains how to use the installer's advanced configuration options to control log rotation and retention.

Log rotation policies

The following policies control log rotation and log expiration on the PM server in the INSTALL_DIR/logs directory:

• Size-based log rotation for the current date. When an original log file reaches the configured maximum size, it is rotated and a suffix is added.

• Number of rotated files: 7. You can configure this value using the logs_rotate_count parameter in PACKAGE_DIR/group_vars/all/vars.yml.

• Maximum file size: 200 MB. You can configure this value using the logs_rotate_maxsize and logs_rotate_size parameters in PACKAGE_DIR/group_vars/all/vars.yml.

  If the log file size exceeds the Maximum file size or Number of rotated files (by default 1,4 GB), log data is removed from the central server but remains on the source server.

• Log compression. All files older than the current day are automatically compressed with gzip.

• Log cleanup. All files older than 14 days are automatically removed from the aggregation server. This behaviour is configured before the Master server installation using the logs_retention parameter in the installer's PACKAGE_DIR/group_vars/all/vars.yml file. If not specified, the default value is 13, which means that files older than 14 days are removed.

Log rotation configuration

The following parameters are optional during configuration. You can use them to control log retention according to specific data storage policies.

• logs_retention: the number of rotated files of the same log file. Default: 7.

  For manual reconfiguration, edit INSTALL_DIR/logrotate/logrotate.d/<service_name>.conf:

  $ vi /opt/workfusion/logrotate/logrotate.d/<service_name>.conf # under wfuser
  
  # Modify line below
  ...
  rotate 7
  ...

• logs_rotate_maxsize: the maximum size of each rotated file. Default: 200 MB.

  For manual reconfiguration, edit INSTALL_DIR/logrotate/logrotate.d/<service_name>.conf:

  $ vi /opt/workfusion/logrotate/logrotate.d/wf_aggregated_logs.conf # under wfuser
  
  # Modify both lines below
  ...
  size 200M
  ...

Service log configuration

Logs are rotated daily or when an original log file reaches the configured maximum size. Logs are deleted when the total size of archived files exceeds 10 GB or when they are older than 14 days.

Supported components are as follows:

Component name	Service name	Configure log level approach
Control Tower	control-tower	Profile
Workspace	workspace	Profile
Workspace API	workspace-api	Profile
Worker Management Service	worker-management-service	Explicit
Task Dispatcher Service	task-dispatcher-service	Explicit
Bot Manager	bot-manager	Explicit
Bot Agent	bot-agent	Explicit
OCR rest	ocr-rest	Explicit
OCR worker	ocr-worker	Explicit
AutoML Gateway Service	automl-gateway-service	Explicit
AutoML Model Service	automl-model-service	Explicit
AutoML Model Management Service	automl-model-management-service	Explicit

The following parameters are optional and use default values unless specified otherwise.

Parameter	Default value	Description
LOGGING_PROFILE	dev	Defines the logging profile used at startup. If another profile is set in ZooKeeper, the ZooKeeper value is used. Allowed values are dev and prod.
CONSOLE_LOG_PATTERN, FILE_LOG_PATTERN	%d{${LOG_DATEFORMAT_PATTERN:-yyyy-MM-dd HH:mm:ss.SSS}} ${LOG_LEVEL_PATTERN:-%-5p} ${PID:- } [%t] %logger{36} : %m%n}	Describes the log format. For more information about log patterns, see the instruction.
LOGS_FILE_PATH	logs	Defines the path to log files relative to the installation folder. The full default path is <install-dir>/service-name/logs.
LOGS_MAX_FILE_SIZE	100MB	Specifies the maximum size of a log file. When the size is exceeded, the application creates a new log file and writes logs to it. The old file is renamed, and its index is increased. The value can be specified in bytes, kilobytes, megabytes, or gigabytes by adding the KB, MB, or GB respectively. Very small or very large values affect performance.
LOGS_MAX_HISTORY	14	Defines the number of days during for which logs are stored. Older logs are deleted.
LOGS_TOTAL_SIZE_CAP	10GB	Defines the maximum total size of all log files. If this limit is exceeded, the oldest logs are deleted.

Manual reconfiguration

This approach changes values for all services installed on a machine.

$ vi /opt/workfusion/environment.sh         # Override any of the variables listed below and save the changes.
...
export LOGS_FILE_PATH=new_logs/
...
$ wfmanager restart <service-name>            # Restart the service to use a new value.

To change a value for one service only, follow the steps below:

For Control Tower and Workspace:

$ vi /opt/workfusion/<service-name>/bin/setenv.sh      # Open the Tomcat setenv.sh file and add the required environment variable.
...
 -DLOGS_MAX_FILE_SIZE=10MB \
...
$ wfmanager restart <service-name>                                    # Restart the service to use a new value.

For other services:

$ vi /opt/workfusion/supervisord/apps/<service-name>.ini   # Open the supervisord file for the service and change it.
# Add the required variable to JAVA_OPTS.
...
JAVA_OPTS="-DLOGS_FILE_PATH=new_logs/"
...
$ wfmanager restart <service-name>                                    # Restart the service to use a new value.