Understanding index.mapping.total_fields.limit in OpenSearch/ElasticSearch

Posted on

Sometimes, you can get the associated error Limit of total fields [1000] has been exceeded

I will explain what it is and how to fix it.
You can find that error in OpenSaerch/ElasticSearch logs /var/log/opensearch or /var/log/elasticsearch
For example, in the screenshot, you can see that error:

In OpenSearch and Elasticsearch, the number of fields in an index is governed by the index.mapping.total_fields.limit setting. This parameter sets the maximum number of fields allowed in an index mapping, and exceeding this limit will cause indexing operations to fail.
Default Limit:
The default value for index.mapping.total_fields.limit is 1,000 fields per index. This includes:
  • Explicitly defined fields in the mappings.
  • Dynamic fields that are created during document indexing.
Adjusting the Limit:
If you need more fields, you can increase the limit by updating the index settings. However, be cautious, as having too many fields can impact cluster performance (e.g., higher memory usage and slower queries).
Update the limit in Dev-Tools:
  1. For an Existing Index:
PUT /your-index-name/_settings
{
  "index.mapping.total_fields.limit": 2000
}
2. If you want to update the limit automatically when creating an index, use this index template for your index:
PUT _template/your-template-name
{
  "index_patterns": ["*"],
  "settings": {
    "index.mapping.total_fields.limit": 2000
  }
}

The post Understanding index.mapping.total_fields.limit in OpenSearch/ElasticSearch appeared first on SOC Prime.

Related Posts