Index prefixes in elasticsearch

[gmarouli]

Problem Description

In elasticsearch when we perform operations that change the structure of an index (reindexing, searchable snapshots, shrink) we use prefixes to create the new name of the index since an index cannot be renamed. Examples of said prefixes are:

• shrink-<xxxx>-: used for shrunk indices
• restored-: used for fully mounted searchable snapshots
• partial-: used for partially mounted searchable snapshots
• downsample-<fixed-interval>-: used for downsampled indices
• .ds-: used for data stream backing indices
• .fs-: used for data stream failure indices
• .migrated-: used for upgraded data streams

Many of the above can be also combined, for example a backing index that was downsampled, then shrunk, the became a searchable snapshot could have the following prefix: restored-shrink-abcd-downsample-2h-.ds-.

The prefixes have allowed us to facilitate a lot of functionalities but they are causing some issues when they are combined with limited user permissions.

User permissions can be expressed by regexes, for example logs*. This gives a user access to the data stream logs. If this data stream gets upgraded and it's name changed to .migrated-logs then the user will lose access.

This is usually addressed by aliases, so after the original data stream or index is deleted we use the original name as an alias. This solves the majority of the access issues, as long as the alias can be used to access the data.

However, there are operations though that require an index name and this would break. This a known issue of ILM this is why we do not recommend setting up an ILM policy when you have limited permissions.

Disclaimer
This issue is meant to record this behaviour and track it. Solving this is not an easy task and we believe it is not unlikely to require atomic index renaming which is not currently supported.

Here we list some of the discussions we already had.

Can we account for these prefixes?
Not ideal, these prefixes can be combined in different ways so it’s not as straightforward to include them.

Can we avoid them altogether?
Atomic renames could reduce their usage a lot for the 1-1 cases, ILM, data stream lifecycle and reindexing. But not when user wants to preserve both.

Can we convert them to suffixes?
This could have the same regex issues, but it’s likely that they would be less prominent considering how regexes prefer prefixed patterns for efficiency. However, suffixes are more likely to expose incomplete indices during searching for example when a user searches using a wildcard pattern logs*.

Steps to Reproduce

### Speed up ILM
PUT _cluster/settings
{
  "persistent": {
    "indices.lifecycle.poll_interval": "1s"
  }
}

### PUT limited role
PUT _security/role/my-role
{
  "description": "Grants access only to other ds.",
  "cluster": ["all"],
  "indices": [
    {
      "names": [ "my-*" ],
      "privileges": ["manage","all"]
    }
  ]
}

### PUT limited user
POST _security/user/other-user
{
  "password" : "some-pass",
  "roles" : [ "my-role" ],
  "full_name" : "Jack Nicholson",
  "email" : "[email protected]",
  "metadata" : {
    "intelligence" : 7
  }
}

### Set policy as limited user
PUT _ilm/policy/shrink
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_docs": "1"
          },
          "shrink" : {
            "number_of_shards": 1
          }
        }
      },
      "warm": {
        "actions": {
          "set_priority": {
            "priority": 50
          }
        }
      }
    }
  }
}

### PUT index template template as limited user
PUT _index_template/my_template
{
  "index_patterns": ["my-index*"],
  "template":{
    "settings": {
      "number_of_shards": 5,
      "lifecycle.name": "shrink",
      "lifecycle.rollover_alias":"my-index"
    }
  }
}

### PUT index
PUT my-index-0000001
{
  "aliases": {"my-index": {}}
}

### Index a doc to trigger a rollover
PUT http://localhost:9200/my-index/_doc/1
{
  "mary": "test"
}

### GET indices will show you the security exception
GET http://localhost:9200/shrink-*-my-index-0000001/_ilm/explain
{
 .......
"step_info": {
        "type": "security_exception",
        "reason": "action [indices:admin/settings/update] is unauthorized for user [other-user] with effective roles [my-role] on indices [shrink-lt80-my-index-0000001], this action is granted by the index privileges [manage,all]"
      },
.........
}

Related issues: elastic/kibana#208362

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)