Wilson Mar bio photo

Wilson Mar


Calendar YouTube Github


Minions to collect data from each server

US (English)   Norsk (Norwegian)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   Napali   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean


This section describes how to install, configure, and use the Beats component within the Elastic Stack, previously called ELK stack before Beats was added in 2016.

Beats is the collective term for Elastic’s utilities that run on servers to read logs from a variety of sources.

Beats are small, lightweight (written in Golang)

  • Filebeat collects and sends text log files.
  • Metricbeat collects and sends operating system and application data.
  • Packetbeat collects and sends network monitoring data.
  • Winlogbeat collects and sends Windows Event logs data
  • Libbeat collects and sends custom data defined in Golang programs.
  • Heartbeat
  • Auditbeat

Beats usually sends its data to a Logstash receiver, but they can send directly to ElasticSearch database.

Alternatives to get bits onto a server:

A) Ansible

B) Chef

C) Microsoft SCCM

D) Direct download

Beats Marketing

The product marketing page for LogStash is at:


Install on Ubuntu

PROTIP: Beats is written in the Go Language (Golang), which compiles to a static binary containing its own VM. So there is no need to install a run-time such as JVM to run Java.

  1. Add the Elastic public signing key to server:

    wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

  2. Install on Ubuntu:

    apt-get update && apt-get install filebeat

  3. Look at the tail end (last few lines) in the system log:

    tail /var/logs/sys.log

  4. Configure the Filebeat configuration file from “/var/log/*.log” for all files to “/var/log/syslog.log” plus add “document_type: syslog”:

    sed /etc/filebeat/filebeat.yml “/var/log/*.log” “/var/log/syslog.log \n document_type: syslog”

    See https://www.elastic.co/guide/logstash/current/index.html


Download Beats

  1. https://www.elastic.co/downloads/beats


Before forwarding, Logstash can parse and normalize varying schema and formats.

  • Reading
  • Filtering
  • Enhancing
  • Forwarding

Metricbeat modules

  • System Logs
  • Apache web server
  • NginX
  • HAProxy

  • MongoDB
  • MySQL
  • PostgreSQL
  • Redis

  • Zookeeper (Puppet)


Based on this

NOTE: Although Windows machines have Perfmon built-in, Beats collects and forwards the data.

  1. Download and unzip.

    File winlogbeat.full.yml contains ALL specifications.

    File winlogbeat.template.es2x.yml should be ignored since you’re not using v2.

    NOTE: Formats changed in v5, so Python scripts/migrate_beat_config_1_x_to_5_0.py is provided to migrate.

  2. Edit the winlogbeat.yml configuration file to keep verbosity low and discard debug entries:

    - name: Application
      ignore_older: 72h
      level: critical, error, warning
    - name: Security
    - name: System
  3. Add tags so each shipper can be identified when logs are consolidated:

    tags: ["us-east-01"]"
  4. Add fields to specify which environment:

      globo_environment: production
  5. Specify the output destination as a Logstash server (not Elasticsearch server):

      hosts: [""]
  6. Save the file.
  7. Verify whether the server designated can be reached. In PowerShell:

    Invoke-WebRequest -Method PUT -InFile .\winlogbeat.template.json -Uri

  8. Install as a service within PowerShell CLI:


  9. Run in PowerShell:

    winlogbeat.exe -c winlogbeat.yml

  10. Start in PowerShell:

    start-service winlogbeat

  11. View logs in PowerShell:

    Get-Content .\logs\winlogbeat -Wait

    Errors will be reported if Logstash has not been setup.


Beats in Logstash

File beats.conf



This is one of a series on Elastic Stack and monitoring:

  1. Elastic Stack ecosystem of people, websites, tutorials
  2. Elastic Stack architecture and installation
  3. Elastic Scaling (the database engine)
  4. Elastic Query (via REST API)
  5. Elastic Kibana (the visualization engine, like Grafana)
  6. Elastic Logstash to assemble and filter data from Beats
  7. Elastic Beats to collect data from servers