Skip to content

Rules Retriever Failing with Unknown NamedWriteable #126071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
bean710 opened this issue Apr 1, 2025 · 3 comments
Open

Rules Retriever Failing with Unknown NamedWriteable #126071

bean710 opened this issue Apr 1, 2025 · 3 comments
Assignees
@kderusso
Labels
>bug priority:high A label for assessing bug priority to be used by ES engineers :Search Relevance/Ranking Scoring, rescoring, rank evaluation. Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch

Comments

@bean710
Copy link

bean710 commented Apr 1, 2025

Elasticsearch Version

8.17.4

Installed Plugins

No response

Java Version

bundled

OS Version

Cloud

Problem Description

When using the rules retriever, the query will fail on at least one shard. It seems to be inconsistent and will switch between which shards fail (when there is more than one on the searched index) and will sometimes apply the rules scoring and sometimes will not.

Steps to Reproduce

This appears to be reproducible with a brand new rules set and a brand new content index.

Rules:

PUT _query_rules/my-rules
{
  "rules": [
    {
      "rule_id": "1",
      "type": "pinned",
      "actions": {
        "ids": ["6759e60f09a922ba4dbac10d"]
      },
      "criteria": [
        {
          "type": "contains",
          "values": ["search"],
          "metadata": "user_query"
        }
      ]
    }
  ]
}

Query:

POST search-es-blog/_search?error_trace=true&explain=true
{
  "from": 0,
  "size": 10,
  "retriever": {
    "rule": {
      "rank_window_size": 10000,
      "match_criteria": {
        "user_query": "search"
      },
      "ruleset_ids": [
        "my-rules"
      ],
      "retriever": {
        "standard": {
          "query": {
            "match_all": {}
          }
        }
      }
    }
  },
  "_source": false
}

My index was web crawling the Elastic blog, but the actual content of the index does not seem to matter.

Logs (if relevant)

Stack Trace:

org.elasticsearch.ElasticsearchException$1: Unknown NamedWriteable [org.elasticsearch.search.rank.RankDoc][query_rule_rank_doc]
	at [email protected]/org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:706)
	at [email protected]/org.elasticsearch.action.search.SearchPhaseExecutionException.guessRootCauses(SearchPhaseExecutionException.java:161)
	at [email protected]/org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:688)
	at [email protected]/org.elasticsearch.ElasticsearchException.generateFailureXContent(ElasticsearchException.java:634)
	at [email protected]/org.elasticsearch.rest.RestResponse.build(RestResponse.java:200)
	at [email protected]/org.elasticsearch.rest.RestResponse.<init>(RestResponse.java:159)
	at [email protected]/org.elasticsearch.rest.RestResponse.<init>(RestResponse.java:120)
	at [email protected]/org.elasticsearch.rest.action.RestActionListener.onFailure(RestActionListener.java:54)
	at [email protected]/org.elasticsearch.rest.action.RestCancellableNodeClient$1.onFailure(RestCancellableNodeClient.java:97)
	at [email protected]/org.elasticsearch.tasks.TaskManager$1.onFailure(TaskManager.java:215)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75)
	at [email protected]/org.elasticsearch.action.DelegatingActionListener.onFailure(DelegatingActionListener.java:32)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations$RunBeforeActionListener.onFailure(ActionListenerImplementations.java:346)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75)
	at [email protected]/org.elasticsearch.action.DelegatingActionListener.onFailure(DelegatingActionListener.java:32)
	at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onFailure(ContextPreservingActionListener.java:40)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75)
	at [email protected]/org.elasticsearch.action.DelegatingActionListener.onFailure(DelegatingActionListener.java:32)
	at [email protected]/org.elasticsearch.action.search.TransportSearchAction$SearchResponseActionListener.onFailure(TransportSearchAction.java:1952)
	at [email protected]/org.elasticsearch.action.search.TransportSearchAction$1.lambda$onFailure$1(TransportSearchAction.java:524)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations$RunAfterActionListener.onResponse(ActionListenerImplementations.java:300)
	at [email protected]/org.elasticsearch.tasks.TaskManager$1.onResponse(TaskManager.java:203)
	at [email protected]/org.elasticsearch.tasks.TaskManager$1.onResponse(TaskManager.java:197)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations$RunBeforeActionListener.onResponse(ActionListenerImplementations.java:336)
	at [email protected]/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:97)
	at [email protected]/org.elasticsearch.action.search.ClearScrollController.lambda$closeContexts$7(ClearScrollController.java:168)
	at [email protected]/org.elasticsearch.core.AbstractRefCounted$1.closeInternal(AbstractRefCounted.java:125)
	at [email protected]/org.elasticsearch.core.AbstractRefCounted.decRef(AbstractRefCounted.java:77)
	at [email protected]/org.elasticsearch.action.support.RefCountingRunnable.close(RefCountingRunnable.java:113)
	at [email protected]/org.elasticsearch.core.Releasables.close(Releasables.java:35)
	at [email protected]/org.elasticsearch.core.Releasables.closeExpectNoException(Releasables.java:57)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations$2.run(ActionListenerImplementations.java:51)
	at [email protected]/org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:217)
	at [email protected]/org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:97)
	at [email protected]/org.elasticsearch.action.ActionListenerResponseHandler.handleResponse(ActionListenerResponseHandler.java:49)
	at [email protected]/org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1500)
	at [email protected]/org.elasticsearch.transport.InboundHandler.doHandleResponse(InboundHandler.java:434)
	at [email protected]/org.elasticsearch.transport.InboundHandler.handleResponse(InboundHandler.java:383)
	at [email protected]/org.elasticsearch.transport.InboundHandler.executeResponseHandler(InboundHandler.java:150)
	at [email protected]/org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:125)
	at [email protected]/org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:98)
	at [email protected]/org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:822)
	at [email protected]/org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:125)
	at [email protected]/org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:97)
	at [email protected]/org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:62)
	at [email protected]/org.elasticsearch.transport.netty4.Netty4MessageInboundHandler.channelRead(Netty4MessageInboundHandler.java:55)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at [email protected]/io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:107)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at [email protected]/io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1515)
	at [email protected]/io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1378)
	at [email protected]/io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1427)
	at [email protected]/io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
	at [email protected]/io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
	at [email protected]/io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at [email protected]/io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at [email protected]/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at [email protected]/io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
	at [email protected]/io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at [email protected]/io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:796)
	at [email protected]/io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:697)
	at [email protected]/io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:660)
	at [email protected]/io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at [email protected]/io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
	at [email protected]/io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: java.lang.IllegalArgumentException: Unknown NamedWriteable [org.elasticsearch.search.rank.RankDoc][query_rule_rank_doc]
	at org.elasticsearch.common.io.stream.NamedWriteableRegistry.throwOnUnknownWritable(NamedWriteableRegistry.java:151)
	at org.elasticsearch.common.io.stream.NamedWriteableRegistry.getReader(NamedWriteableRegistry.java:126)
	at org.elasticsearch.common.io.stream.NamedWriteableRegistry.getReader(NamedWriteableRegistry.java:109)
	at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:56)
	at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:33)
	at org.elasticsearch.index.query.RankDocsQueryBuilder.lambda$new$0(RankDocsQueryBuilder.java:46)
	at org.elasticsearch.common.io.stream.StreamInput.readArray(StreamInput.java:1090)
	at org.elasticsearch.index.query.RankDocsQueryBuilder.<init>(RankDocsQueryBuilder.java:46)
	at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:57)
	at org.elasticsearch.common.io.stream.NamedWriteableAwareStreamInput.readNamedWriteable(NamedWriteableAwareStreamInput.java:33)
	at org.elasticsearch.search.builder.SubSearchSourceBuilder.<init>(SubSearchSourceBuilder.java:80)
	at org.elasticsearch.common.io.stream.StreamInput.readCollection(StreamInput.java:1319)
	at org.elasticsearch.common.io.stream.StreamInput.readCollectionAsList(StreamInput.java:1191)
	at org.elasticsearch.search.builder.SearchSourceBuilder.<init>(SearchSourceBuilder.java:242)
	at org.elasticsearch.common.io.stream.StreamInput.readOptional(StreamInput.java:1117)
	at org.elasticsearch.common.io.stream.StreamInput.readOptionalWriteable(StreamInput.java:1107)
	at org.elasticsearch.search.internal.ShardSearchRequest.<init>(ShardSearchRequest.java:287)
	at org.elasticsearch.transport.RequestHandlerRegistry.newRequest(RequestHandlerRegistry.java:63)
	at org.elasticsearch.transport.InboundHandler.handleRequest(InboundHandler.java:262)
	at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:118)
	at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:98)
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:822)
	at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:125)
	at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:97)
	at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:62)
	at org.elasticsearch.transport.netty4.Netty4MessageInboundHandler.channelRead(Netty4MessageInboundHandler.java:55)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:107)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1515)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1378)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1427)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:796)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:697)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:660)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.lang.Thread.run(Thread.java:1575)
@bean710 bean710 added >bug needs:triage Requires assignment of a team area label labels Apr 1, 2025
@bean710 bean710 changed the title Rules Retriever Failing Rules Retriever Failing with Unknown NamedWriteable Apr 1, 2025
@jbaiera jbaiera added :Search Foundations/Search Catch all for Search Foundations and removed needs:triage Requires assignment of a team area label labels Apr 4, 2025
@elasticsearchmachine elasticsearchmachine added the Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch label Apr 4, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-foundations (Team:Search Foundations)

@andreidan andreidan added :Search Relevance/Ranking Scoring, rescoring, rank evaluation. and removed :Search Foundations/Search Catch all for Search Foundations labels Apr 7, 2025
@elasticsearchmachine elasticsearchmachine added Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch and removed Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch labels Apr 7, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@javanna javanna added the priority:high A label for assessing bug priority to be used by ES engineers label Apr 7, 2025
@kderusso
Copy link
Member

kderusso commented Apr 8, 2025

Script to duplicate:

PUT index-001/_doc/1
{
    "text": "patio"
}

PUT index-001/_doc/2
{
    "text": "catio"
}

PUT index-002/_doc/3
{
    "text": "balcony"
}

PUT index-002/_doc/4
{
    "text": "overhang"
}

GET /index-001,index-002/_search
{
  "retriever": {
    "standard": {
      "query": {
        "query_string": {
          "query": "patio or balcony"
        }
      }
    }
  }
}

PUT _query_rules/my-ruleset
{
    "rules": [
      {
        "rule_id": "my-rule",
        "type": "pinned",
        "criteria": [
          {
            "type": "exact",
            "metadata": "query_string",
            "values": ["patio or balcony"]
          }
        ],
        "actions": {
            "ids": [ "2" ]
        }
      }
    ]
}


GET /index-001,index-002/_search?error_trace=true
{
  "retriever": {
    "rule": {
      "retriever": {
        "standard": {
          "query": {
            "query_string": {
              "query": "patio or balcony"
            }
          }
        }
      },
      "match_criteria": {
        "query_string": "patio or balcony"
      },
      "ruleset_ids": [ "my-ruleset" ]
    }
  }
}

This script will break in 8.17.4, and partially break in 8.18.0/9.0/8.x/main:

{
  "took": 35,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 1,
    "skipped": 0,
    "failed": 1,
    "failures": [
      {
        "shard": 0,
        "index": "index-001",
        "node": "WWihbswMSmmu6WuI5hymcQ",
        "reason": {
          "type": "illegal_argument_exception",
          "reason": "Unknown NamedWriteable [org.elasticsearch.search.rank.RankDoc][query_rule_rank_doc]"
        }
      }
    ]
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 0.6931471,
    "hits": [
      {
        "_index": "index-002",
        "_id": "3",
        "_score": 0.6931471,
        "_source": {
          "text": "balcony"
        }
      }
    ]
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kderusso kderusso

Labels
>bug priority:high A label for assessing bug priority to be used by ES engineers :Search Relevance/Ranking Scoring, rescoring, rank evaluation. Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@javanna @jbaiera @andreidan @bean710 @elasticsearchmachine @kderusso