Elasticsearch - Scenario Based  Question - Data Streams, ILM, Component Template, Index Template

Create an Index Lifecycle Management Policy
that will keep the data `hot` for first 5 minutes
Rollsover the after 5 minutes to `warm` Phase
Data Stays in warm phase for 3 minutes from rollover
Data Stays in `cold` phase for 6 minutes from rollover
Finally Delete the Data in delete phase

Create a component template named `logs-component-template`
that uses the `logs-index-lifecycle-policy` ILM Policy

Create an Index Template named `logs-index-template`
that looks for pattern `logs-*-*`
meant for Data Stream

Finally start the data stream by sending below data
POST logs-august-2022/_doc/
{
  "@timestamp": "2022-08-22T11:06:07.000Z", 
  "message": "Login successful"
}

Task

PUT _ilm/policy/logs-index-lifecycle-policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "5m"
          }
        }
      },
      "warm": {
        "min_age": "5m",
        "actions": {}
      },
      "cold": {
        "min_age": "8m",
        "actions": {}
      },
      "delete": {
        "min_age": "14m",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

Solution

# Create a component template named `logs-component-template`
# that uses the `logs-index-lifecycle-policy` ILM Policy

PUT _component_template/logs-component-template
{
  "template": {
    "settings": {
      "index.lifecycle.name": "logs-index-lifecycle-policy"
    }
  }
}

Solution

# Create an Index Template named `logs-index-template`
# that looks for pattern `logs-*-*`
# meant for Data Stream

PUT _index_template/logs-index-template
{
  "index_patterns": ["logs-*-*"],
  "data_stream": { },
  "composed_of": [ "logs-component-template" ]
  
}

Solution

POST logs-august-2022/_doc/
{
  "@timestamp": "2022-08-22T11:06:07.000Z",
  "message": "Login successful"
}


GET logs-august-2022/_search
GET logs-*-*/_search

Solution

THANKS

FOR

WATCHING

Elasticsearch-Scenario-Based-Question-Data-Streams

By Deepak Dubey

Elasticsearch-Scenario-Based-Question-Data-Streams

  • 864