10.0

Edit

INT

Logstash

The Logstash service is running on the Integration server and contains six pipelines:

  • logstash-spa receives log events from all Filebeat services, places them into the install-dir/logs directory and sends to the ElasticSearch filebeat-* index.
  • logstash-automl-extract receives log events from running AutoML extract models and places them into the INSTALL_DIR/logs/bep_master/BEP-MASTER_HOSTNAME/automl-extract directory.
  • logstash-bep-logs receives log events from BEP services and workers and places them into the /logs/bep/ directory.
  • logstash-bep-metrics receives BEP services metrics and sends them into a DB for further processing by Analytics.
  • logstash-metrics receives system metrics from all hosts with the installed Metricbeat service and sends them to a DB for further processing by Analytics.
  • logstash-heartbeat receives Heartbeat status logs from all the hosts and components with installed Heartbeat and sends them to the heartbeat-* index in ElasticSearch and to /logs/SERVICE_NAME/SERVICE_HOST/COMPONENT_NAME/DATE/heartbeat-status.log
  • Software owner: Elasticsearch BV
  • Management: wfmanager {start stop restart status} logstash
  • Log files: INSTALL_DIR/supervisord/log/logstash.log
  • Startup scripts:
    • INSTALL_DIR/supervisord/apps/logstash.ini
    • INSTALL_DIR/logstash/bin/logstash
  • Configuration files: INSTALL_DIR/logstash/logstash.conf
  • Default ports:
    • tcp 4567 (logstash-spa) 4568 (logstash-automl-extract)
    • 4569 (logstash-metrics) 4570 (logstash-bep-logs)
    • 15072 (logstash-bep-metrics) 4571 (logstash-heartbeat)

ElasticSearch

  • Software owner: ElasticSearch BV
  • Management: wfmanager {start stop restart status} elasticsearch
  • Log files: INSTALL_DIR/supervisord/log/elastisearch.log
  • Startup scripts:
    • INSTALL_DIR/supervisord/apps/elasticsearch.ini
    • INSTALL_DIR/elasticsearch/bin/
  • Configuration files: INSTALL_DIR/elasticsearch/config/elasticsearch.yml
  • Default ports: tcp 9200

Vault

  • Software owner: Hashicorp Vault
  • Management: wfmanager {start stop restart status} vault:*
  • Log files:
    • Access logs: INSTALL_DIR/supervisord/log/vault-server.out.log
    • Error logs: INSTALL_DIR/supervisord/log/vault-server.err.log
  • Startup scripts:
    • INSTALL_DIR/supervisord/apps/vault.ini
    • INSTALL_DIR/vault/vault
  • Configuration files: INSTALL_DIR/vault/config/server.hcl
  • Default ports: tcp 8200

Zookeeper

  • Software owner: Apache Software Foundation
  • System requirements: https://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html#sc_systemReq
  • Management: wfmanager {start stop restart status} zookeeper
  • Log files: INSTALL_DIR/supervisord/log/zookeeper.log
  • Startup scripts:
    • INSTALL_DIR/supervisord/apps/zookeeper.ini
    • INSTALL_DIR/zookeeper/bin/zkServer.sh
  • Configuration files:
    • Service directory: INSTALL_DIR/zookeeper/
    • General zookeeper configuration: INSTALL_DIR/zookeeper/zoo.cfg
  • Default ports: tcp 2181

Rabbitmq

  • Software owner: Pivotal Software, Inc.
  • System requirements: 4 CPUs, 4 GB RAM, 2 GB free space
  • Management: wfmanager {start stop restart status} rabbitmq
  • Log files: INSTALL_DIR/supervisord/log/rabbitmq.log
  • Startup scripts:
    • INSTALL_DIR/supervisord/apps/rabbitmq.ini
    • INSTALL_DIR/rabbitmq/sbin/rabbitmq-server
  • Configuration files:
    • Service directory: INSTALL_DIR/rabbitmq
    • General rabbitmq configuration: INSTALL_DIR/rabbitmq/etc/rabbitmq/rabbitmq.config
  • Default ports: tcp 5672

Minio

  • Software owner: Minio, Inc
  • Management: wfmanager (start \ stop \ restart \ status) minio
  • Log files: INSTALL_DIR/supervisord/log/minio.log
  • Startup scripts: INSTALL_DIR/supervisord/apps/minio.ini
  • Configuration files: INSTALL_DIR/minio/etc/config.json
  • Default Ports: tcp 9000

Nexus

  • Software owner: Sonatype
  • Management: wfmanger (start \ stop \ restart \ status) nexus
  • Log files: INSTALL_DIR/supervisord/log/nexus.*.log
  • Data Files: INSTALL_DIR/nexus/sonatype-work/nexus/
  • Installation directories: INSTALL_DIR/nexus/
  • Configuration files: INSTALL_DIR/nexus/conf/nexus.properties
  • Default Ports: 80/443

Task Dispatcher Service

  • Software owner: The Apache Software Foundation
  • Management: wfmanager {start stop restart status} task-dispatcher-service
  • Log files: INSTALL_DIR/supervisord/log/task-dispatcher-service.log
  • Startup scripts: INSTALL_DIR/supervisord/apps/task-dispatcher-service.ini
  • Configuration files: INSTALL_DIR/task-dispatcher-service/task-dispatcher-service.yml
← PreviousNext →