Open
Description
Description
Changing indices, data streams, component templates, index templates, ingest pipelines and so on is often interdependent - e.g. I need to make sure the component template and index template is ready before I adjust an ingest pipeline to extract new fields and route data to a new place.
However, it's currently not possible to apply this kind of choices in an atomic way - instead, it's necessary to go one by one.
This has some downsides:
- Race conditions: It's possible for multiple clients to change config concurrently, which can lead to half-overwritten configurations. It's very difficult to move from one consistent state to another
- Intermediate states: Since each config change becomes active immediately, a client has to be careful when applying multiple changes (e.g. what if a data stream rolls over after I changed one component template, but not the other one)
- Rollbacks: In the presence of bugs, one out of a bunch of changes might fail - in this case, it's very hard to reason about the current half-applied state and how to roll it back to the last consistent version
All of this is hard to fix outside of Elasticsearch. It would be nice if a new primitive or API for this could be offered.
(Removing brainstorming notes until we decide on what exactly we'd need here)