{"id":636,"date":"2022-01-27T09:00:57","date_gmt":"2022-01-27T09:00:57","guid":{"rendered":"https:\/\/www.canchito-dev.com\/public\/blog\/?p=636"},"modified":"2022-01-30T14:11:29","modified_gmt":"2022-01-30T14:11:29","slug":"collect-metrics-with-metricbet","status":"publish","type":"post","link":"http:\/\/www.canchito-dev.com\/public\/blog\/2022\/01\/27\/collect-metrics-with-metricbet\/","title":{"rendered":"Collect Metrics with Metricbet"},"content":{"rendered":"<h1>Collect Metrics with Metricbeat<\/h1>\n<div class=\"perfect-pullquote vcard pullquote-align-full pullquote-border-placement-left\"><blockquote><p><\/p>\n<p>Learn how you could use Metricbeat to monitor your servers by collecting metrics from the system and services running on the server.<\/p>\n<p><\/p><\/blockquote><\/div>\n<div><a class=\"donate-with-crypto\" href=\"https:\/\/commerce.coinbase.com\/checkout\/faf64f90-2e80-46ee-aeba-0fde14cbeb46\"><br \/>\nBuy Me a Coffee<br \/>\n<\/a><br \/>\n<script src=\"https:\/\/commerce.coinbase.com\/v1\/checkout.js?version=201807\">\n  <\/script><\/div>\n<div class=\"titlepage\">\n<div>\n<div>\n<h2 class=\"title\">Introduction<\/h2>\n<\/div>\n<\/div>\n<\/div>\n<p style=\"text-align: justify;\">Until now, we have covered several components of the Elastic Stack. For instance, we <a href=\"http:\/\/www.canchito-dev.com\/public\/blog\/2021\/12\/26\/introduction-to-elastic-stack\/\">learn about Elasticsearch for storing the data that we collect and how to deploy it, Kibana as a Web UI for visualizing the collected data<\/a>, <a href=\"http:\/\/www.canchito-dev.com\/public\/blog\/2022\/01\/02\/elastic-stack-beats\/\">Filebeat for collecting data from our cluster<\/a>, and last we <a href=\"https:\/\/www.canchito-dev.com\/public\/blog\/2022\/01\/16\/getting-started-with-logstash\/\">saw what Logstash can do<\/a>. Now, it is time to learn about Metricbeat and how it helps you monitor your servers by collecting metrics from the system and services running on the server.<\/p>\n<h2>Overview<\/h2>\n<p style=\"text-align: justify;\">Metricbeat is an Elastic Beat, and as such, it\u2019s based on the <code class=\"literal\">libbeat<\/code> framework. Periodically collect metrics from your server&#8217;s operating system and from services running on the server with the help of this lightweight shipper. Metricbeat takes the metrics and statistics that it collects and ships them to the output that you specify, such as Elasticsearch or Logstash.<\/p>\n<p style=\"text-align: justify;\">Metricbeat consists of modules and metricsets. A <em>module<\/em> in Metricbeat, defines the basic logic for collecting data from a specific service, such as Kafka, Apache, MySQL, and so on. It is here, where the details about the service are specified. These include how to connect, how often to collect metrics, and which metrics to collect.<\/p>\n<p style=\"text-align: justify;\">Each module has one or more metricsets. A\u00a0<em>metricset<\/em> is the part of the module that fetches and structures the data. Rather than collecting each metric as a separate event, metricsets retrieve a list of multiple related metrics in a single request to the remote system.<\/p>\n<h2>Deploying Metricbeat in Docker<\/h2>\n<p style=\"text-align: justify;\">Let&#8217;s start by adding a folder which will have Metricbeat&#8217;s files. The changes in the project should be highlighted.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-highlight=\"22-24,29\">elastic-stack-demo\r\n  +- elasticsearch-single-node-cluster\r\n       +- elasticsearch\r\n       |    +- Dockerfile-elasticsearch-single-node\r\n       |    +- elasticsearch-single-node.yml\r\n       +-filebeat\r\n       |    +- Dockerfile\r\n       |    +- filebeat-to-elasticsearch.yml\r\n       |    +- filebeat-to-logstash.yml\r\n       +-kibana\r\n       |    +- Dockerfile-kibana-single-node\r\n       |    +- kibana-single-node.yml\r\n       +-logstash\r\n       |    +- config\r\n       |    |    +- logstash.yml\r\n       |    |    +- pipelines.yml\r\n       |    +- pipeline\r\n       |    |    +- beats-example.conf\r\n       |    |    +- data-stream-example.conf\r\n       |    |    +- output.conf\r\n       |    +- Dockerfile\r\n       +-metricbeat\r\n       |    +- Dockerfile\r\n       |    +- metricbeat.yml\r\n       +- .env\r\n       +- docker-compose-es-single-node.yml\r\n       +- docker-compose-filebeat-to-elasticseach.yml\r\n       +- docker-compose-filebeat-to-logstash.yml\r\n       +- docker-compose-logstash.yml\r\n       +- docker-compose-metricbeat.yml\r\n<\/pre>\n<p style=\"text-align: justify;\">The first file we will be creating is the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"yaml\">Dockerfile<\/code>. Create it under <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">elastic-stack-single-node-cluster\/metricbeat\/<\/code>, and paste the following code:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"yaml\">ARG ELK_VERSION\r\nFROM docker.elastic.co\/beats\/metricbeat:${ELK_VERSION}\r\n\r\n# add custom configuration\r\nCOPY --chown=root:metricbeat metricbeat.yml \/usr\/share\/metricbeat\/metricbeat.yml<\/pre>\n<p style=\"text-align: justify;\">The file has nothing extraordinary. it is just specifying the base image and copying the configuration <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">YAML<\/code> file for Metricbeat. This configuration file looks like this:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"yaml\">########################## Metricbeat Configuration ###########################\r\n# You can find the full configuration reference here:\r\n# https:\/\/www.elastic.co\/guide\/en\/beats\/metricbeat\/index.html\r\n\r\n#============================  Config Reloading ===============================\r\n# Config reloading allows to dynamically load modules. Each file which is\r\n# monitored must contain one or multiple modules as a list.\r\nmetricbeat.config.modules:\r\n  # Glob pattern for configuration reloading\r\n  path: ${path.config}\/modules.d\/*.yml\r\n  # Period on which files under path should be checked for changes\r\n  reload.period: 10s\r\n  # Set to true to enable config reloading\r\n  reload.enabled: false\r\n\r\n# Maximum amount of time to randomly delay the start of a metricset. Use 0 to\r\n# disable startup delay.\r\nmetricbeat.max_start_delay: 10s\r\n\r\n#============================== Autodiscover ===================================\r\n# Autodiscover allows you to detect changes in the system and spawn new modules\r\n# as they happen.\r\nmetricbeat.autodiscover:\r\n  providers:\r\n    - type: docker\r\n      hints.enabled: true\r\n\r\n#==========================  Modules configuration =============================\r\nmetricbeat.modules:\r\n  #-------------------------------- Docker Module --------------------------------\r\n  - module: docker\r\n    metricsets:\r\n      - \"container\"\r\n      - \"cpu\"\r\n      - \"diskio\"\r\n      - \"healthcheck\"\r\n      - \"info\"\r\n      - \"memory\"\r\n      - \"network\"\r\n    hosts: [\"unix:\/\/\/var\/run\/docker.sock\"]\r\n    period: 10s\r\n    enabled: true\r\n    # If set to true, collects metrics per core.\r\n    cpu.cores: true\r\n\r\n# ================================= Processors =================================\r\n# Processors are used to reduce the number of fields in the exported event or to\r\n# enhance the event with external metadata. This section defines a list of\r\n# processors that are applied one by one and the first one receives the initial\r\n# event:\r\n#\r\n#   event -&gt; filter1 -&gt; event1 -&gt; filter2 -&gt;event2 ...\r\n#\r\n# The supported processors are drop_fields, drop_event, include_fields,\r\n# decode_json_fields, and add_cloud_metadata.\r\nprocessors:\r\n  # The following example enriches each event with docker metadata, it matches\r\n  # container id from log path available in `source` field (by default it expects\r\n  # it to be \/var\/lib\/docker\/containers\/*\/*.log).\r\n  - add_docker_metadata: ~\r\n  # The following example enriches each event with host metadata.\r\n  - add_host_metadata: ~\r\n  # The following example enriches each event with process metadata using\r\n  # process IDs included in the event.\r\n  - add_process_metadata: ~\r\n\r\n# ================================== Outputs ===================================\r\n# Configure what output to use when sending the data collected by the beat.\r\n# ---------------------------- Elasticsearch Output ----------------------------\r\noutput.elasticsearch:\r\n  # Boolean flag to enable or disable the output module.\r\n  enabled: true\r\n  # Array of hosts to connect to.\r\n  # Scheme and port can be left out and will be set to the default (http and 9200)\r\n  # In case you specify and additional path, the scheme is required: http:\/\/localhost:9200\/path\r\n  # IPv6 addresses should always be defined as: https:\/\/[2001:db8::1]:9200\r\n  hosts: ['elasticsearch-demo:9200']\r\n\r\n# ================================= Dashboards =================================\r\n# These settings control loading the sample dashboards to the Kibana index. Loading\r\n# the dashboards are disabled by default and can be enabled either by setting the\r\n# options here, or by using the `-setup` CLI flag or the `setup` command.\r\nsetup.dashboards.enabled: true\r\n\r\n# =================================== Kibana ===================================\r\n# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.\r\n# This requires a Kibana endpoint configuration.\r\nsetup.kibana:\r\n  # Kibana Host\r\n  # Scheme and port can be left out and will be set to the default (http and 5601)\r\n  # In case you specify and additional path, the scheme is required: http:\/\/localhost:5601\/path\r\n  # IPv6 addresses should always be defined as: https:\/\/[2001:db8::1]:5601\r\n  host: \"kibana-demo:5601\"\r\n\r\n# ================================== Logging ===================================\r\n# There are four options for the log output: file, stderr, syslog, eventlog\r\n# The file output is the default.\r\n# Sets log level. The default log level is info.\r\n# Available log levels are: error, warning, info, debug\r\nlogging.level: info\r\n\r\n# ============================= X-Pack Monitoring ==============================\r\n# Metricbeat can export internal metrics to a central Elasticsearch monitoring\r\n# cluster.  This requires xpack monitoring to be enabled in Elasticsearch.  The\r\n# reporting is disabled by default.\r\n\r\n# Set to true to enable the monitoring reporter.\r\nmonitoring.enabled: true\r\n\r\n# Uncomment to send the metrics to Elasticsearch. Most settings from the\r\n# Elasticsearch output are accepted here as well.\r\n# Note that the settings should point to your Elasticsearch *monitoring* cluster.\r\n# Any setting that is not set is automatically inherited from the Elasticsearch\r\n# output configuration, so if you have the Elasticsearch output configured such\r\n# that it is pointing to your Elasticsearch monitoring cluster, you can simply\r\n# uncomment the following line.\r\nmonitoring.elasticsearch:\r\n  # Array of hosts to connect to.\r\n  # Scheme and port can be left out and will be set to the default (http and 9200)\r\n  # In case you specify and additional path, the scheme is required: http:\/\/localhost:9200\/path\r\n  # IPv6 addresses should always be defined as: https:\/\/[2001:db8::1]:9200\r\n  #hosts: [\"elasticsearch-demo:9200\"]\r\n\r\n# =============================== HTTP Endpoint ================================\r\n# Each beat can expose internal metrics through a HTTP endpoint. For security\r\n# reasons the endpoint is disabled by default. This feature is currently experimental.\r\n# Stats can be access through http:\/\/localhost:5066\/stats . For pretty JSON output\r\n# append ?pretty to the URL.\r\n# Defines if the HTTP endpoint is enabled.\r\nhttp.enabled: true\r\n\r\n# The HTTP endpoint will bind to this hostname, IP address, unix socket or named pipe.\r\n# When using IP addresses, it is recommended to only use localhost.\r\nhttp.host: metricbeat-demo\r\n\r\n# Port on which the HTTP endpoint will bind. Default is 5066.\r\nhttp.port: 5066\r\n<\/pre>\n<p style=\"text-align: justify;\">As you can see, we have included the description of each configuration option. Hopefully, it will be easier to understand it. However, the main idea behind it, is:<\/p>\n<ul>\n<li style=\"text-align: justify;\">Enable the autodiscover feature, based on hints. Autodiscover allows you to track applications and monitor services as they start running. The hints system looks for hints in Kubernetes Pod annotations or Docker labels that have the prefix\u00a0<code class=\"literal\">co.elastic.metrics<\/code>. As soon as the container starts, Metricbeat will check if it contains any hints and launch the proper config for it.<\/li>\n<li>Enable docker module, so that we can monitor other containers in the same network.<\/li>\n<li style=\"text-align: justify;\">Enable providers, which work by watching for events on the system and translating those events into internal autodiscover events with a common format.<\/li>\n<li style=\"text-align: justify;\">Send the collected data to Elasticsearch for indexing.<\/li>\n<li style=\"text-align: justify;\">Automatically create predefined dashboards and load them into Kibana.<\/li>\n<li style=\"text-align: justify;\">Export internal metrics to a central Elasticsearch monitoring cluster, by enabling x-pack monitoring. In our case, we will be using the same cluster.<\/li>\n<li>Enable experimental HTTP endpoint, which exposes internal metrics.<\/li>\n<\/ul>\n<p>Now, we create a separate docker-compose file under <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">elastic-stack-single-node-cluster\/<\/code> and name it <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">docker-compose-metricbeat.yml<\/code>.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"yaml\">version: '3.9'\r\nservices:\r\n  metricbeat-demo:\r\n    hostname: metricbeat-demo\r\n    container_name: metricbeat-demo\r\n    build:\r\n      context: .\/metricbeat\r\n      dockerfile: Dockerfile\r\n      args:\r\n        - ELK_VERSION=${ELK_VERSION}\r\n    ports:\r\n      - 5366:5066\r\n    # Need to override user so we can access the log files, and docker.sock\r\n    user: root\r\n    cap_add:\r\n      - SYS_PTRACE\r\n      - DAC_READ_SEARCH\r\n    volumes:\r\n      - \/proc:\/hostfs\/proc:ro\r\n      - \/sys\/fs\/cgroup:\/hostfs\/sys\/fs\/cgroup:ro\r\n      - \/:\/hostfs:ro\r\n      - \/var\/run\/docker.sock:\/var\/run\/docker.sock\r\n      - \/var\/run\/dbus\/system_bus_socket:\/var\/run\/dbus\/system_bus_socket:ro\r\n      - data_metricbeat_demo:\/usr\/share\/metricbeat\/data\r\n    # disable strict permission checks\r\n    command: [ '-e', '-v', '--strict.perms=false', '-system.hostfs=\/hostfs' ]\r\n    #network_mode: host # Mandatory to monitor HOST filesystem, memory, processes,...\r\n    networks:\r\n      - elastic-stack-service-network\r\n\r\n# Networks to be created to facilitate communication between containers\r\nnetworks:\r\n  elastic-stack-service-network:\r\n    name: elastic-stack-service-network\r\n\r\n# Volumes\r\nvolumes:\r\n  data_metricbeat_demo:\r\n    driver: local\r\n<\/pre>\n<p style=\"text-align: justify;\">When executing Metricbeat in a container, there are some important things to be aware of if you want to monitor the host machine or other containers. For instance,\u00a0Metricbeat\u2019s\u00a0system module\u00a0collects much of its data through the Linux proc filesystem, which is normally located at\u00a0<code class=\"literal\">\/proc<\/code>. Due to containers are isolated as much as possible from the host, the data from the host\u2019s\u00a0<code class=\"literal\">\/proc<\/code> is different than the inside of the container\u2019s <code class=\"literal\">\/proc<\/code>. To solve this, you can mount the host\u2019s <code class=\"literal\">\/proc<\/code> filesystem inside of the container and by using the\u00a0<code class=\"literal\">-system.hostfs=\/hostfs<\/code>\u00a0CLI flag, specify Metricbeat to look inside the <code class=\"literal\">\/hostfs<\/code>\u00a0directory when looking for\u00a0<code class=\"literal\">\/proc<\/code> .<\/p>\n<p style=\"text-align: justify;\">Out of the box, cgroup reporting is enabled for the system process metricset. That is why, you need to mount the host\u2019s cgroup mountpoints within the container and inside the directory specified by the <code class=\"literal\">-system.hostfs<\/code> CLI flag. In this case, it is the <code class=\"literal\">\/hostfs\/sys\/fs\/cgroup<\/code> directory.<\/p>\n<p style=\"text-align: justify;\">If you want to be able to monitor filesystems from the host by using the\u00a0system filesystem metricset, then those filesystems need to be mounted inside of the container. They can be mounted at any location.<\/p>\n<p style=\"text-align: justify;\">When using <code class=\"literal\">-system.hostfs=\/hostfs<\/code>, the\u00a0system network metricset\u00a0uses data from\u00a0<code class=\"literal\">\/hostfs\/proc\/net\/dev<\/code>, otherwise it is from\u00a0<code class=\"literal\">\/proc\/net\/dev<\/code>. Due to Linux namespacing; it is not enough to bind mounting the host\u2019s <code class=\"literal\">\/proc<\/code>\u00a0to\u00a0<code class=\"literal\">\/hostfs\/proc<\/code>. The only way to make this file contain the host\u2019s network devices is to use the\u00a0<code class=\"literal\">--net=host<\/code> flag. In docker-compose, this flag is represented by <code class=\"literal\">network_mode: host<\/code>. Nevertheless, you can only set either the network mode or connect Metricbeat to a docker&#8217;s isolated virtual network. For this time, we will leave it in a virtual network so that it can access the other containers.<\/p>\n<p style=\"text-align: justify;\">Since we are using a Linux based operating system, more privileges need to be granted to Metricbeat if we ought to use system socket metricset. By default, docker disables the capabilities to read files from\u00a0<code class=\"literal\">\/proc<\/code> that are an interface to internal objects owned by other users. The capabilities needed to read all these files are <code class=\"literal\">sys_ptrace<\/code>\u00a0and\u00a0<code class=\"literal\">dac_read_search<\/code>.<\/p>\n<p style=\"text-align: justify;\">Great. We are ready to start Metricbeat, by executing the following command:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\">$ docker-compose -f docker-compose-metricbeat.yml up -d --build<\/pre>\n<p style=\"text-align: justify;\">If you go to <a href=\"http:\/\/localhost:5601\/app\/dashboards#\/list\">Analytics &gt; Dashboards<\/a> and look for a dashboard called <code class=\"literal\">[Metricbeat System] Host overview ECS<\/code>. Click it and you will see an overview of the host&#8217;s metrics:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" data-attachment-id=\"644\" data-permalink=\"http:\/\/www.canchito-dev.com\/public\/blog\/2022\/01\/27\/collect-metrics-with-metricbet\/metricbeat_system_dashboard__host_overview_ecs\/\" data-orig-file=\"http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs.png\" data-orig-size=\"5102,2616\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"CANCHITO-DEV: [Metricbeat System] Host overview ECS\" data-image-description=\"\" data-image-caption=\"&lt;p&gt;CANCHITO-DEV: [Metricbeat System] Host overview ECS&lt;\/p&gt;\n\" data-large-file=\"http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs-1024x525.png\" class=\"aligncenter wp-image-644 size-full\" src=\"http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs.png\" alt=\"CANCHITO-DEV: [Metricbeat System] Host overview ECS\" width=\"5102\" height=\"2616\" srcset=\"http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs.png 5102w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs-300x154.png 300w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs-1024x525.png 1024w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs-768x394.png 768w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs-1536x788.png 1536w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs-2048x1050.png 2048w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_system_dashboard__host_overview_ecs-624x320.png 624w\" sizes=\"auto, (max-width: 5102px) 100vw, 5102px\" \/><\/p>\n<p>And finally, since we also enabled the docker module, we can also see a dashboard (<code class=\"literal\">[Metricbeat Docker] Overview ECS<\/code>) showing us an overview of all our docker containers&#8217; metrics:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" data-attachment-id=\"643\" data-permalink=\"http:\/\/www.canchito-dev.com\/public\/blog\/2022\/01\/27\/collect-metrics-with-metricbet\/metricbeat_docker_dashboard_overview_ecs\/\" data-orig-file=\"http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs.png\" data-orig-size=\"5114,2158\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"CANCHITO-DEV: [Metricbeat Docker] Overview ECS\" data-image-description=\"\" data-image-caption=\"&lt;p&gt;CANCHITO-DEV: [Metricbeat Docker] Overview ECS&lt;\/p&gt;\n\" data-large-file=\"http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs-1024x432.png\" class=\"aligncenter wp-image-643 size-full\" src=\"http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs.png\" alt=\"CANCHITO-DEV: [Metricbeat Docker] Overview ECS\" width=\"5114\" height=\"2158\" srcset=\"http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs.png 5114w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs-300x127.png 300w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs-1024x432.png 1024w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs-768x324.png 768w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs-1536x648.png 1536w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs-2048x864.png 2048w, http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metricbeat_docker_dashboard_overview_ecs-624x263.png 624w\" sizes=\"auto, (max-width: 5114px) 100vw, 5114px\" \/><\/p>\n<h2>Clean Up<\/h2>\n<p>To do a complete clean up, execute this command:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\">$ docker-compose -f docker-compose-es-single-node.yml -f docker-compose-filebeat-to-elasticseach.yml -f docker-compose-filebeat-to-logstash.yml -f docker-compose-logstash.yml-f docker-compose-metricbeat.yml down -v<\/pre>\n<h2 style=\"text-align: justify;\">Summary<\/h2>\n<p style=\"text-align: justify;\">In this post, we learn about Metricbeat and how to configure and deploy it in a dockerized environment.<\/p>\n<p style=\"text-align: justify;\">Please feel free to contact us. We will gladly response to any doubt or question you might have. In the mean time, you can download the source code from our official <a href=\"https:\/\/github.com\/canchito-dev\/elastic-stack-demo\">GitHub<\/a> repository.<\/p>\n<p style=\"text-align: justify;\">\n","protected":false},"excerpt":{"rendered":"<p>Learn how you could use Metricbeat to monitor your servers by collecting metrics from the system and services running on the server.<\/p>\n","protected":false},"author":1,"featured_media":645,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[94,88,84,96,1],"tags":[104,63,75,115,89,108],"class_list":["post-636","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-beat","category-elastic-stack","category-elk","category-metricbeat","category-uncategorized","tag-beat","tag-docker","tag-docker-compose","tag-elastic-stask","tag-elk","tag-metricbeat"],"aioseo_notices":[],"jetpack_featured_media_url":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/metrics-logo-color.png","jetpack_shortlink":"https:\/\/wp.me\/p8EwXo-ag","jetpack-related-posts":[{"id":590,"url":"http:\/\/www.canchito-dev.com\/public\/blog\/2022\/01\/02\/elastic-stack-beats\/","url_meta":{"origin":636,"position":0},"title":"Introduction to Elastic Stack Beats","author":"canchitodev","date":"January 2, 2022","format":false,"excerpt":"How is information sent to Elasticsearch? The answer is simple, using Beats or Logstash. In this post, we will give a brief introduction to Beats.","rel":"","context":"In &quot;Auditbeat&quot;","block_context":{"text":"Auditbeat","link":"http:\/\/www.canchito-dev.com\/public\/blog\/category\/elk\/auditbeat\/"},"img":{"alt_text":"CANCHITO-DEV: Beats basic diagram","src":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/beats_diagram.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/beats_diagram.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/beats_diagram.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/beats_diagram.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/beats_diagram.png?resize=1050%2C600 3x"},"classes":[]},{"id":647,"url":"http:\/\/www.canchito-dev.com\/public\/blog\/2022\/01\/31\/know-if-your-service-is-available-with-heartbeat\/","url_meta":{"origin":636,"position":1},"title":"Know if your Service is Available with Heartbeat","author":"canchitodev","date":"January 31, 2022","format":false,"excerpt":"Learn how Heartbeat periodically checks the status of your services and determine whether they are available. All within a dockerized enviroment.","rel":"","context":"In &quot;Beat&quot;","block_context":{"text":"Beat","link":"http:\/\/www.canchito-dev.com\/public\/blog\/category\/elk\/beat\/"},"img":{"alt_text":"CANCHITO-DEV: [Heartbeat] HTTP Monitoring","src":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/heartbeat_dashboard_http_monitoring.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/heartbeat_dashboard_http_monitoring.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/heartbeat_dashboard_http_monitoring.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/heartbeat_dashboard_http_monitoring.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/heartbeat_dashboard_http_monitoring.png?resize=1050%2C600 3x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/heartbeat_dashboard_http_monitoring.png?resize=1400%2C800 4x"},"classes":[]},{"id":564,"url":"http:\/\/www.canchito-dev.com\/public\/blog\/2021\/12\/26\/introduction-to-elastic-stack\/","url_meta":{"origin":636,"position":2},"title":"Introduction to Elastic Stack","author":"canchitodev","date":"December 26, 2021","format":false,"excerpt":"Hello friends! In this post, we will give you a small introduction to Elastic Stack including all the products that build it.","rel":"","context":"In &quot;Beat&quot;","block_context":{"text":"Beat","link":"http:\/\/www.canchito-dev.com\/public\/blog\/category\/elk\/beat\/"},"img":{"alt_text":"CANCHITO-DEV: Elastic Stack architecture in Docker","src":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/docker_elastic_stack_architecture.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/docker_elastic_stack_architecture.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/docker_elastic_stack_architecture.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/docker_elastic_stack_architecture.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/docker_elastic_stack_architecture.png?resize=1050%2C600 3x"},"classes":[]},{"id":665,"url":"http:\/\/www.canchito-dev.com\/public\/blog\/2022\/02\/05\/audit-the-activities-of-users-and-processes-on-your-systems-with-auditbeat\/","url_meta":{"origin":636,"position":3},"title":"Audit the activities of users and processes on your systems with Auditbeat","author":"canchitodev","date":"February 5, 2022","format":false,"excerpt":"Get to know Auditbeat and learn how it can help you by auditing the activities of the users and processes on your systems. All within a dockerized enviroment.","rel":"","context":"In &quot;Auditbeat&quot;","block_context":{"text":"Auditbeat","link":"http:\/\/www.canchito-dev.com\/public\/blog\/category\/elk\/auditbeat\/"},"img":{"alt_text":"CANCHITO-DEV: Kibana's Management > Stack Monitoring Complete","src":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/02\/stack_monitoring_whole_architecture.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/02\/stack_monitoring_whole_architecture.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/02\/stack_monitoring_whole_architecture.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/02\/stack_monitoring_whole_architecture.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/02\/stack_monitoring_whole_architecture.png?resize=1050%2C600 3x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/02\/stack_monitoring_whole_architecture.png?resize=1400%2C800 4x"},"classes":[]},{"id":654,"url":"http:\/\/www.canchito-dev.com\/public\/blog\/2022\/01\/31\/capturing-the-network-traffic-with-packetbeat\/","url_meta":{"origin":636,"position":4},"title":"Capturing the network traffic with Packetbeat","author":"canchitodev","date":"January 31, 2022","format":false,"excerpt":"Discover how Packetbeat sniffs the traffic between your servers, parses the application-level protocols on the fly, and correlates the messages into transactions. All within a dockerized enviroment.","rel":"","context":"In &quot;Beat&quot;","block_context":{"text":"Beat","link":"http:\/\/www.canchito-dev.com\/public\/blog\/category\/elk\/beat\/"},"img":{"alt_text":"CANCHITO-DEV: [Packetbeat] Overview ECS","src":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/packetbeat_dashboard_overview_ecs.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/packetbeat_dashboard_overview_ecs.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/packetbeat_dashboard_overview_ecs.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/packetbeat_dashboard_overview_ecs.png?resize=700%2C400 2x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/packetbeat_dashboard_overview_ecs.png?resize=1050%2C600 3x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2022\/01\/packetbeat_dashboard_overview_ecs.png?resize=1400%2C800 4x"},"classes":[]},{"id":588,"url":"http:\/\/www.canchito-dev.com\/public\/blog\/2021\/12\/31\/deploying-filebeat-in-docker\/","url_meta":{"origin":636,"position":5},"title":"Deploying Filebeat in docker","author":"canchitodev","date":"December 31, 2021","format":false,"excerpt":"Learn about Filebeat and how it interact with the rest of the Elastic Stack components while you deploy it using docker.","rel":"","context":"In &quot;Beat&quot;","block_context":{"text":"Beat","link":"http:\/\/www.canchito-dev.com\/public\/blog\/category\/elk\/beat\/"},"img":{"alt_text":"CANCHITO-DEV: Filebeat Overview","src":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/filebeat_overview.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/filebeat_overview.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/filebeat_overview.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.canchito-dev.com\/public\/blog\/wp-content\/uploads\/2021\/12\/filebeat_overview.png?resize=700%2C400 2x"},"classes":[]}],"jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/posts\/636","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/comments?post=636"}],"version-history":[{"count":8,"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/posts\/636\/revisions"}],"predecessor-version":[{"id":650,"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/posts\/636\/revisions\/650"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/media\/645"}],"wp:attachment":[{"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/media?parent=636"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/categories?post=636"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.canchito-dev.com\/public\/blog\/wp-json\/wp\/v2\/tags?post=636"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}