The "mapper-size" plugin allows you to track the size of documents in your Elasticsearch/OpenSearch indices. Here's how you can enable it and query for large log messages:
Enable "mapper-size" Plugin:
Before you can query for large log messages, you need to enable the "mapper-size" plugin for your Elasticsearch/OpenSearch indices. You can do this by updating the mapping of your index. In the Kibana/OpenSearch Dev Tools console, run the following command:
PUT your-index-name
{
"mappings": {
"_doc": {
"_size": {
"enabled": true
}
}
}
}
Replace your-index-name with the actual name of your log index. This command enables the "mapper-size" for that specific index.
Query for Large Log Messages:
Now that "mapper-size" is enabled, you can query for large log messages. You can use Kibana/OpenSearch's Discover or Visualize feature to create a search for large log messages. Here's an example Kibana/OpenSearch Query DSL for finding log messages larger than a specific size (e.g., 1MB):
{
"query": {
"range": {
"_size": {
"gte": 1000000
}
}
}
}
This query will return log messages with a size greater than or equal to 1MB. You can adjust the value (1,000,000 in this example) to fit your specific size threshold.
Save and Visualize:
After creating your query, you can save it in Kibana/OpenSearch and create visualizations or dashboards to monitor large log messages effectively.
By following these steps, you can enable the "mapper-size" plugin, configure your index to track document sizes and create queries in Kibana/OpenSearch to find and analyze large log messages in your Elasticsearch/OpenSearch indices.
Daily Ingestion
If you wanted to have the size field available for all ingested data going forward then you would need to make some changes to the index templates, for example:
PUT _template/your-template-name
{
"order": 5,
"index_patterns": ["filebeat-*"],
"mappings": {
"_doc": {
"_size": {
"enabled": true
}
}
}
}
If you need assistance updating your templates to include this change simply reach today to a member of our team via live chat or via support@logit.io.
What's next:
Learn now to understand better your Log Volumes
Learn about Elasticsearch Ingest Pipelines