OpenSearch Split Index API

Posted on

The Split Index API in OpenSearch is a useful feature that allows you to split an existing index into multiple smaller indices. This can be particularly valuable when you want to improve performance, scale your index, or rebalance the data without re-ingesting it.

What is the Split Index API?

The Split Index API enables you to take a large index and divide it into smaller ones, increasing the number of primary shards. The number of primary shards in the new (split) index must be a multiple of the existing primary shards. This API is often used when:
  • Your existing index is too large for its shard configuration.
  • You need to scale out horizontally by increasing shards.
  • You want to rebalance the data across more nodes for performance or capacity reasons.

Requirements for Splitting an Index

  1. Read-Only State: The index must be in a read-only state.
  2. Routing Allocation: You need to set the routing allocation for the split operation.
  3. Multiple of Shards: The new number of shards must be a multiple of the original.
Example:
You have an index named my_index with 2 primary shards. You want to split it into 4 shards.
 
1. Make the Index Read-Only
First, set the index to read-only mode:
PUT /my_index/_settings
{
  "settings": {
    "index.blocks.write": true
  }
}

2. Split the Index
Now use the Split Index API to create a new index my_index_split with 4 primary shards:

POST /my_index/_split/my_index_split
{
  "settings": {
    "index.number_of_shards": 4,
    "index.routing.allocation.require._name": "node_name" 
  }
}
Here:
  • my_index is the source index.
  • my_index_split is the new target index.
  • index.number_of_shards is set to 4 (a multiple of 2).
  • index.routing.allocation.require._name ensures the split happens on a specific node (optional).
3. Remove Read-Only Block
Once the split is successful, remove the read-only block from the index:
PUT /my_index/_settings
{
  "settings": {
    "index.blocks.write": false
  }
}

Validation and Confirmation

To verify the split, you can check the shard count for the new index:

GET /my_index_split/_settings
You’ll see the new index with 4 primary shards as expected.Key Points
  • Splitting an index is only allowed for closed indices.
  • The number of shards must be a multiple of the current count.
  • This is a great way to scale without reindexing or ingesting data again.

When to Use Split Index

  • You initially created an index with too few shards, and now it’s underperforming.
  • You want to balance your cluster’s load and improve query performance.

The post OpenSearch Split Index API appeared first on SOC Prime.

Related Posts