No error when create index if there are nodes that have files and no files

[johtani]

Elasticsearch version: 5.2.2

Description of the problem including expected versus actual behavior:

The situation is below:

• Using mapping_char filter with mappings_path, e.g. fileA.txt
• Elasticsearch has 2 nodes and a node has fileA.txt and another node does not have fileA.txt

Then, user can create an index with mapping_char filter if the master node gets the request and has fileA.txt, whether the number of replicas is 0 or not.
But, user cannot create an index if the master node gets the request and does not have fileA.txt

We should notify an error to user there are some strange nodes.

And if create an index with the number of shards = 2 and the number of replicas = 0 with 2 nodes cluster, RELOCATING is repeated indefinitely...

[lewischen1123]

Interested in picking up this task. i want to have a try thx

[gwbrown]

I've confirmed that this issue is still present in 7.10.0, at least the part about the file being required on the master node and odd errors occurring if the index is created and some nodes don't have the file.

I'm not sure if there's a great way to address this. I'm going to mark this team-discuss to see if we have any ideas for how it would be possible to alleviate this issue, as ensuring files are the same across nodes is a problem I've seen come up a few times.

[gwbrown]

We discussed this in the core/infra sync today.

Unfortunately, this problem is difficult to solve. Verifying that all nodes have the file would be a complex and possibly slow process, and by the time we have the mappings we've already passed most of the opportunities to do something at node start time.

The decision we reached was that we'll review the error messages that occur in both 1) the case that the master node does not have the file and rejects the index creation, and 2) the case that the index is created and one or more data nodes don't have the necessary file. As long as these error messages make the problem clear, ensuring the filesystem is in the correct state on each node is the responsibility of the cluster administrator, at least for now. We'd like to improve this in the future, but this is a complicated and difficult problem to solve in a way that would work well in all cases.

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[martijnvg]

Re-assigned the search relevance analysis label, given that this is an issue with the mapping char filter. If this is the wrong label, then please let me know.