Detect when copying to a non-existent field when dynamic mappings are disabled

[Mikep86]

Description

We should be able to detect when a user is attempting to use copy_to to copy to a non-existent field when dynamic mappings are disabled. For example:

PUT test-index
{
  "mappings": {
    "dynamic": false,
    "properties": {
      "test_field": {
        "type": "text",
        "copy_to": "missing_field"
      }
    }
  }
}

Currently, this request succeeds even though nothing will ever be copied to missing_field. We should throw a 400 error in this case to indicate to the user that their mappings are invalid.

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)

[elasticsearchmachine]

Pinging @elastic/search-relevance (Team:Search - Relevance)

[elasticsearchmachine]

Pinging @elastic/ent-search-eng (Team:SearchOrg)

[Obolrom]

@Mikep86 could you please assign this issue to me?

[Mikep86]

@Obolrom Great to hear you're interested in addressing this issue! Please feel free to submit a PR for this, but we don't officially assign tasks to community members. I highly doubt someone at Elastic will address this before you do though :)

[henriquepaes1]

Hello! @Obolrom let me know if you're still interested in solving this issue. Otherwise, I would like to work on it.

[Obolrom]

@henriquepaes1
I have a lot of work. So yeah, take it)

[henriquepaes1]

Hello @Mikep86! I'm currently working on this issue. I found the "copy-to" verifications in the FieldMapper class in order to insert the non-existent field verification. However, I have a question: what is the expected behavior when dynamic mappings are enabled?

[Mikep86]

@henriquepaes1 We expect the request to succeed in that case because the target field can be dynamically created. Note that I expect the solution to involve FieldTypeLookup, as that is where all the mappers can be validated with respect to one another.

[henriquepaes1]

@Mikep86 I appreciate the prompt reply. I understood the expected behavior and what the solution should envolve, I'll work on that later today.

[likithy]

@henriquepaes1 We expect the request to succeed in that case because the target field can be dynamically created. Note that I expect the solution to involve FieldTypeLookup, as that is where all the mappers can be validated with respect to one another.

@Mikep86 can you please review this and approve i tried to work on enhancement

[ioanatia]

We do want to make sure that we pay attention to index templates.

Example:

PUT _index_template/template_1
{
  "index_patterns": ["test-index*"],
  "template": {
    "mappings": {
      "properties": {
        "missing_field": {
          "type": "text"
        }
      }
    }
  }
}

PUT test-index
{
  "mappings": {
    "dynamic": false,
    "properties": {
      "test_field": {
        "type": "text",
        "copy_to": "missing_field"
      }
    }
  }
}

GET test-index

PUT test-index/_doc/12
{
  "test_field": "hello"
}

# this will return results
GET test-index/_search
{
  "query": {
    "match": {
      "missing_field": "hello"
    }
  }
}

In this case the test-index index uses an index template that specifies a mapping for missing_field so we should not fail the PUT test-index request.

[henriquepaes1]

Hey @ioanatia thanks for pointing this out! I tested my solution with that case and while it worked fine, it raised some questions.

1- What is the difference between using the MappingLookup class and the FieldTypeLookup @Mikep86 mentioned a few comments above?
2- Would you help me writing a test case for the cenario you described? I didn't found any test in the CopyToMapperTests class that uses templates.

[kunal5042]

Hi Mikep86,

Can I work on this issue, I'm very new in terms on contributing to open-source, as far as I've understood, this is not resolved yet and has been marked with good first issue label. Kindly let me know if it's okay for me to pick this up.

Thanks & Regards

[chandu05-dev]

Hi Mikep86,

Can I work on this issue, I'm very new in terms on contributing to open-source, as far as I've understood, this is not resolved yet and has been marked with good first issue label. Kindly let me know if it's okay for me to pick this up.

Thanks & Regards

Hey Kunal,
hope you are doing well. I am new to opensource development and wanted to ask how has your journey been so far. I would love to collaborate with you so that I can learn from you.

[priorigratia]

@henriquepaes1 are you still working on this issue? If not, would love to take this up.

[BinhMike]

Hi Mikep86,

Can I work on this issue. As I've understood, this is not resolved yet and has been marked with issue label. Kindly let me know if it's okay for me to pick this up.

Thanks & Regards