Edit

Platform Monitor configuration

Running agent

Prerequisites: JRE 8

To run the agent, change <VERSION> to the appropriate value:

java -Xmx512m -jar status-agent-<VERSION>.jar --spring.config.location=classpath:/application.yml # With configuration file inside .jar
java -Xmx512m -jar status-agent-<VERSION>.jar --spring.config.location=/opt/status/application.yml # With configuration file placed on the server

Logs

Logs are created relatively to application home directory in logs.

Absolute path to the logs dir can be specified via -DLOG_DIR=.

Configuration

The default configuration (no need to specify in custom config file):

info:
  company: WorkFusion
  build.version: "@project.version@"
  build.timestamp: "@timestamp@"
  
pools:
  executor:
     queuecapacity: 100
     corepoolsize: 2
     maxpoolsize: 100
      
reporter:
  console:
    enable: true
    interval: 1
  
credentials: []
groups: []
alerts: []
components: []
checks: []

Simple custom configuration example:

groups:
  - { id: vds, name: "VDS Services"}

components:
  - id: gateway
    name: VDS Gateway
    group: vds
    description: "Version %gateway.info.version%"
    alerts: [ gateway ]
 
alerts:
  - id: gateway
    triggers:
      - { type: performance,  metric: gateway.info.time, gte: 10000 }
      - { type: major_outage, metric: gateway.info.http.code, neq: 200 }   
 
checks:
  - id: gateway.info
    type: http
    expression: "http://localhost:8080/project-info"
    evaluate:
      - { id: http.json, arg: version, type: string, metric_name: version}
 

General settings

Concurrency settings

pools:
  executor:
     maxpoolsize: Number of threads in primary executor pool
     queuecapacity: Number of tasks in queue for thread pool (grows when pool is saturated)
     corepoolsize: Default number of threads

Unirest settings are used for HTTP client configuration:

unirest:
  concurrency:
    maxTotal: 200
    maxPerRoute: 20
  timeouts:
    connection: 10000
    socket: 60000

Reporter settings

reporter:
  console:
    enabled: true # Whether print all metrics to console
    interval: 1   # Interval to print metrics (in minutes)
  slf4j:
    enabled: true # Whether log all metrics using slf4j logger
    interval: 1   # Interval to print metrics (in minutes)
  mongo:
    enabled: true # Whether dump all metrics to MongoDB (required for metrics history and proactive checks)
    source: Mongo # Datasource name of MongoDB database
    interval: 1   # Interval to dump ametrics (in minutes)

Model backend

model:
  backend: memory # Supported backend types: memory/mongo/cachet. Memory backend type doesn't have any persistence
  source: Mongo   # Datasource name of MongoDB database. Not required for "memory" and "cachet" backends

Metrics backend

metrics.history:
    source: Mongo            # Datasource name of MongoDB database where all metrics are stored
    cleanup: "0 0 0 * * *"   # Schedule of cleanup for metrics data (once per day)
    days: 40                 # Remove metrics older than 40 days

Environment

This is a key and pair values which can be used as values for some attributes later in config (checks, datasources).

Environment keys can be referenced later via %key%:

environment:
  key1: value1
  key2: value2

Examples

environment:
  ocr_hostname: localhost
  ocr_protocol: http
  ocr_port: 8080
 
checks:
  - id: ocr
    type: http
    expression: "%ocr_protocol%://%ocr_hostname%:%ocr_port%/api/project-info"

Credentials

User and password pairs that can be used within checks.

credentials:
  - id: Id of credentials pair
    user: user name
    password: password

Examples

• Rabbitmq

credentials:
  - id: rabbitmq
    user: admin
    password: password
 
checks:
  - id: queue
    credentials: rabbitmq
    expression: "http://localhost:15672/api/overview"

Datasources

Datasources can be linked within a <Check> so that corresponding check types can retrieve data.

datasources:
  - id: ID of datasource
    type: One of supported drivers
    protocol: Protocol of datasource (http, https)
    host: Hostname or IP
    port: Port of resource
    db: Dabase name
    url: URL for connection
    user: User name
    password: Password

Supported datasource types:

• mongo
• sql
• mysql
• postgresql
• elasticsearch
• http
• tcp

Examples

• Mongo datasource

datasources:
  - id: LocalMongo
    type: mongo
    host: localhost
    db: ocr

• Mysql datasource

- id: LocalMysql
  type: mysql
  url: "jdbc:mysql://localhost:3306/test?characterEncoding=UTF-8&autoReconnect=true&useSSL=false"
  user: root
  password: root

• Postgres datasource

- id: LocalMysql
  type: mysql
  url: "jdbc:postgresql://localhost:5432/postgres"
  user: root
  password: root

• Http datasource

- id: OcrRest
  type: http
  protocol: http
  host: localhost
  port: 8080

Groups

List of component groups

groups:
  - id: Id of group (internal use, can be referenced from components)
    name: Name of group
    collapsed: Collapse the group? 0 = No. 1 = Yes. 2 = If a component is not Operational. Default is 1

Examples

groups:
  - { id: vds.services, name: "VDS Services"}

Components

List of components

components:
  - id: Id of component (internal, can be referenced later)
    name: Name of component
    group: Id of group
    description: Description of component, metric values can be referenced with %metric_id%
    link: URL link for component
    enabled: true/false
    alerts: List of alert ids to react on

Examples:

components:
  - id: vds.gateway
    name: VDS Gateway
    group: vds.services
    description: "Version %vds.gateway.info.http.json.version%"
    alerts: [vds.gateway]

Alerts

Alerts is a scheduled validation of metrics using some evaluation functions.

alerts:
  - id: Id of alert
    description: Description of alert
    dependson: Metric id which should have valid status (used to check underlying status agent proxy). Higher preference than alert check attribute
    enabled: true/false
    templates: List of template objects which replace specific check properties
    triggers:
      - metric: Name of metric to check
        gt: Check that value is greater than
        lt: Check that value is less than
        gte: Check that value is greater than or equal
        lte: Check that value is less than or equal
        eq: Check that value is equal
        neq: Check that value is not equal
        message: Message for this particular metric check
        points: How many points (times) should match condition before trigger (Default is 3 which means that only 3rd failed check will trigger alert).
        dependson: Metric id which should have valid status (used to check underlying status agent proxy).

Examples:

alerts:
  - id: disk.prediction
    checks: 
      - { metric: server.disk.forecast.5m, lte: 0, type: performance, message: "Disk will be used in 5 minutes" }
      - { metric: server.disk.forecast.1h, lte: 0, type: performance, message: "Disk will be used in 1 hour" }
      - { metric: server.disk.forecast.3d, lte: 0, type: performance, message: "Disk will be used in 3 days" }

Checks

Checks is a scheduled action which produces metrics (based on supplied evaluation functions).

checks:
  - id: Id of check
    description: Human-readable description of check purpose
    type: One of supported check types
    file: Use file contents as expression result
    expression: Check type will use this to evaluate metrics
    schedule: Cron-like expression for periodic execution or number of seconds between checks
    tag: Tag which can be used for metric aggregation via "tag.*" functions
    evaluate: List of evaluation functions to get metric values
    templates: List of template objects which replace specific check properties
    enabled: true/false
    dependson: Other check id. Only run this check if dependson check is successful
    parameters: Additional parameters for specific checks (like analysis)
    sslFactory: true/false Use or don't use ssl factory for http-check
    ns: Namespace for evaluated metrics
    readTimeout: check request read timeout in milliseconds, default is 60000 (1 min)

Check types

• http
• http-json
• tcp
• command

Command check type can work in two modes:

• Default mode: run the command internally. This mode doesn't support pipes or redirection.
• Second mode: dump content of expression into a temporary file and execute it.

Example for Windows: the default mode:

- id: command
  type: command
  expression: "curl -Il -s https://www.workfusion.com/"

Example for Windows: with a temporary file:

- id: command
  type: command
  expression: "curl -Il -s https://www.workfusion.com/"
  parameters:
    executable: "cmd.exe /c"

Example for Linux: with a temporary file:

- id: command
  type: command
  expression: "curl -Il -s https://www.workfusion.com/"
  parameters:
    executable: bash

• mongo
• sql
• java
• analysis

Evaluation functions

This function processes check result and extracts additional metrics.

evaluate:
  - id: Id of function
    arg: Argument to function
    file: Use file contents as expression
    type: Object type of metric (default - "long")
    metric: override metric suffix name (default - name of function) (check id and function id are prepended)
    metric_name: override metric name (only check id is prepended)

How metric names are generated

Pattern: <CHECK_ID> + <METRIC_NAME>

Examples:

Metric: vds.gateway.resp

check:
  - id: vds.gateway
    evaluate:
      - id: http.code
        metric_name: resp

Additional patterns when metric_name is not specified: <CHECK_ID> + <FUNCTION_ID>

Metric: vds.gateway.http.code

check:
  - id: vds.gateway
    evaluate:
      - id: http.code

<CHECK_ID> + <FUNCTION_ID> + <METRIC>

Metric: vds.gateway.http.code.resp

check:
  - id: vds.gateway
    evaluate:
      - id: http.code
        metric: resp

Schedule format:

• Cron expression: "*/20 * * * * *". Run every 20 seconds
• Fixed delay in seconds: "20" (Default)

Examples:

• command check

- id: vds.gateway.info
  type: command
  expression: "curl -s http://localhost:8080/project-info"
  schedule: "20"
  evaluate:
    - { id: command.exit }
    - { id: command.output, type: string }

• mongo check

- id: ocr.db
  type: mongo
  expression: "{ dbStats: 1 }"
  schedule: "20"
  source: LocalMongo
  evaluate:
    - { id: mongo.object, arg: dataSize, type: long, metric: size}

• mysql check

- id: mysql.db.status
  type: sql
  expression: "show status like 'Threads_connected'"
  schedule: "20"
  source: LocalMysql
  evaluate:
    - { id: sql.output, arg: Value, type: long, metric: connections }

Evaluation functions

type	id	description	arg	generated metrics
http	http.json	Get response body as JSON and extract object	1) Exact key from JSON object. Examples: "key", "key.child" 2) "register_all" which will register all child key/value pairs to metrics. Used to retrieve stats from proxy Status Agent	default pattern
http	http.response	Get response body as value		default pattern
http	rabbitmq.summary	Get summary from rabbitmq stats.		"messages", "messages_ready", "messages_unacknowledged"
command	command.exit	Exit code of command		default pattern
command	command.output	Output of the command	1) "errors" - only add output to metric when command is not successful	default pattern
command	command.cpu.util	Command to get cpu utilization info. Provides a value of load from 0.0 to 1.0 which takes into account available cores.		load
command	table.value.index	With expression, for example, "cat /proc/meminfo" we can parse output specifying needed args.	1) "0,MemTotal,1" - only returns MemTotal value from table output	memtotal
mongo	mongo.object	Get object from mongo query result	1) Exact key to retrieve. Examples: "key"	default pattern
sql	sql.output	Get output from SQL query	1) Value of specified column Note: Arg can be omitted if result returns single value	default pattern
java	disk.space	Get status from disk	Path which should be checked for disk space information	"total", "free"
http	rpa.manager.hubs	Parses SG hubs JSON		"running", "max"
http	rpa.manager.quotas	Parses SG quotas JSON		"running", "max"
java	expression	Calculates new metric based on expression. Template variables can be used. Metric names for templating are taken from previous evaluation functions results	Expression examples: 1) "1 + 1" 2) "%metric1% + %metric2%"	default pattern
java	metric	Fetches existing metric from registry into result of current check. Can be useful by pairing with subsequent "expression" function	1) "rpa.hubs.all.missing" 2) "rpa.hubs.%hub1_hostname%.missing"	default pattern
java	tag.sum	Aggregation function which will iterate all existing checks with specific tag and sum their results. "metric" is the metric to aggregate "metric_name" is the result metric name	1) "rpa.hub" - tag name
analysis	anomaly.egads	Find anomalies in timeseries data		default pattern
analysis	forecast.regression	Forecast metric value in future	Number of hours in future	default pattern
analysis	trend	Calculate metric trend		default pattern

Dashboards

Dashboard is a collection of metrics which will be displayed on a single page.

dashboards:
  - id: Id of dashboard
    metrics: List of metric names

Example:

dashboards:
  - id: workfusion
    metrics:
      - memory.utilization
      - cpu.spike
      - disk.decreasing

Template engine

Template engine allows creating multiple configuration objects using a single definition.

Example check with two templates:

- type: http
  source: api
  templates:
    - { id: check1, expression: "/api/first" }
    - { id: check2, expression: "/api/second" }

Example above will be parsed and two checks will be created similar to the following definitions:

- id: check1
  type: http
  source: api
  expression: "/api/first"
 
- id: check2
  type: http
  source: api
  expression: "/api/second"

Not all configuration objects and properties are supporting templates. Below is the comprehensive list of what is currently supported.

Configurations with templating support

Check

• id
• description
• expression
• source
• schedule
• file
• ns
• sslFactory
• credentials

Alert

• id
• description
• dependson
• schedule

Command

• id
• type
• schedule
• expression
• source
• credentials
• ns

Substituting variables

For all supported template properties, you can use environment variables that are replaced on the final phase of loading configuration.

Example:

- type: http
  source: api
  templates:
    - { id: check1.%var1%, expression: "/api/first/%var1%" }
    - { id: check2.%var2%, expression: "/api/second/%var2%" }