Step 1: Add AWS Credentials to the Keystore
To securely store your AWS credentials, use the OpenSearch keystore.
Add your AWS credentials to the OpenSearch keystore with the following commands:
./bin/opensearch-keystore add s3.client.default.access_key
./bin/opensearch-keystore add s3.client.default.secret_key
Enter your AWS access_key and secret_key when prompted. OpenSearch will use these stored credentials for S3 interactions.
Step 2: Connect to S3 Bucket
Now that your credentials are stored securely, configure OpenSearch to connect to your S3 bucket for snapshot storage.
Create the S3 repository by using the following command:
PUT _snapshot/my_s3_repository
{
"type": "s3",
"settings": {
"bucket": "my-opensearch-backups",
"region": "us-west-2"
}
}
This sets up the S3 repository in OpenSearch, and OpenSearch will use the credentials from the keystore to interact with AWS.
Step 3: Create a Slack Notification Channel
To receive Slack notifications, you need to set up a notification channel in OpenSearch.
Create an Incoming Webhook in Slack:
- Go to your Slack workspace and navigate to Apps.
- Search for and select Incoming Webhooks.
- Create a new webhook for a channel you want notifications to appear in (e.g., #backups).
- Copy the webhook URL provided by Slack.
Create the Notification Channel in OpenSearch: With the Slack webhook URL in hand, create a Slack notification channel in OpenSearch:
PUT /_plugins/_notifications/channels/my_slack_channel
{
"type": "slack",
"slack": {
"webhook_url": "https://hooks.slack.com/services/your/webhook/url"
}
}
Replace https://hooks.slack.com/services/your/webhook/url with your actual Slack webhook URL.
Step 4: Retrieve the Notification Channel ID
Once the notification channel is created, you can retrieve its ID by listing all notification channels. This is done by running the following query in OpenSearch Dev Tools:
GET /_plugins/_notifications/channels
The response will include a list of channels, each with its id. Use the appropriate channel.id in your snapshot policy configuration.
Step 5: Create Snapshot Management Policy
Now, define a Snapshot Management (SM) policy that schedules snapshots every 8 hours, deletes snapshots older than 365 days, and sends notifications to Slack.
Here’s an example of how to configure the snapshot policy:
POST /_plugins/_sm/policies/my_sm_policy
{
"sm_policy": {
"name": "my_sm_policy",
"description": "my_sm_policy",
"creation": {
"schedule": {
"cron": {
"expression": "0 */8 * * *"
}
}
},
"deletion": {
"schedule": {
"cron": {
"expression": "0 */8 * * *"
}
},
"condition": {
"max_age": "365d",
"min_count": 273,
"max_count": 365
}
},
"snapshot_config": {
"indices": "*",
"include_global_state": true,
"repository": "my_s3_repository"
},
"schedule": {
"interval": {
"period": 1,
"unit": "Minutes"
}
},
"enabled": true,
"notification": {
"channel": {
"id": "my_slack_channel"
},
"conditions": {
"creation": true,
"deletion": false,
"failure": true,
"time_limit_exceeded": false
}
}
}
}
This policy will take snapshots every 8 hours and delete snapshots older than 365 days. Notifications will be sent to Slack for events related to snapshot creation and failure.
The post Creating Snapshot Management Policies with Keystore Integration and Slack Notification Setup appeared first on SOC Prime.
