What's an Elasticsearch mapping issue?

A mapping issue happens when the data you send to your Logit ELK stack doesn't match the the current index templates. This means Elasticsearch does not know what to do with the data and one of two things usually happens here. Elasticsearch will either choose to drop the data you've sent (meaning you won't have any visible logs/data) or it will keep the data but it will not be parsed correctly.
The default index patterns available to you are as follows:

  •  *-*
  • auditbeat-*
  • filebeat-*
  • heartbeat-*
  • logstash-*
  • metricbeat-*
  • packetbeat-*
  • winlogbeat-*

Tip: You can quickly view your current index patterns by launching Kibana for any of your Logit ELK Stacks and choose Management > Index Patterns.

How to fix Elasticsearch mapping issues?

Using Logstash Filters

You can make sure the data sent to Elasticsearch isn't dropped and gets parsed correctly by adding to your Logstash filters. You can access your Logstash filter from the dashboard of any of your stacks by choosing Stack Settings > Logstash Filters. 

Here you are going to add a new index for the data you are sending. An example of how to add a new index via the Logstash filter is below.

if[type] == "CONDITION" {  
   mutate
   {      
      add_field => { "[@metadata][index]" => "YOURINDEXNAME" }
   }
}

What this will do is put all data that matches the "CONDITION" into a new index called “YOURINDEXNAME” in your ELK stack (this can be called anything you choose.)

The condition part allows you to decide which logs will go to the new index, based on your data. 

if[type] == "CONDITION" {

More information on how to change index names and index name limitations can be found in this article.

Using the output section in an Elastic beat configuration

You can also specify the index mapping within the outputs section of an Elasticbeat such as Filebeat or Metricbeat. Locate the configuration file for the beat you are using to send data and navigate to the Logstash output section.

"Index" will specify the name that the collected events write to. The default index would normally be the beat name. For example Filebeat would generate the following indices with a daily timestamp.

[filebeat-]YYYY.MM.DD

What's next?


Did this answer your question?