Skip to content

Updating documentation for text_similarity_reranker for 8.x #126971

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions docs/reference/reranking/semantic-reranking.asciidoc
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
[[semantic-reranking]]
== Semantic re-ranking

preview::[]

[TIP]
====
This overview focuses more on the high-level concepts and use cases for semantic re-ranking. For full implementation details on how to set up and use semantic re-ranking in {es}, see the <<text-similarity-reranker-retriever,reference documentation>> in the Search API docs.
Expand Down Expand Up @@ -87,11 +85,11 @@ To use semantic re-ranking in {es}, you need to:

. *Select and configure a re-ranking model*.
You have the following options:
.. Use the <<inference-example-elastic-reranker,Elastic Rerank>> cross-encoder model via the inference API's {es} service.
.. Use the <<inference-example-elastic-reranker,Elastic Rerank>> model through a preconfigured `.rerank-v1-elasticsearch` endpoint or create a custom one using the inference API's {es} service.
.. Use the <<infer-service-cohere,Cohere Rerank inference endpoint>> to create a `rerank` endpoint.
.. Use the <<infer-service-google-vertex-ai,Google Vertex AI inference endpoint>> to create a `rerank` endpoint.
.. Upload a model to {es} from Hugging Face with {eland-docs}/machine-learning.html#ml-nlp-pytorch[Eland]. You'll need to use the `text_similarity` NLP task type when loading the model using Eland. Then set up an <<inference-example-eland,{es} service inference endpoint>> with the `rerank` endpoint type.
+
+
Refer to {ml-docs}/ml-nlp-model-ref.html#ml-nlp-model-ref-text-similarity[the Elastic NLP model reference] for a list of third party text similarity models supported by {es} for semantic re-ranking.

. *Create a `rerank` endpoint using the <<put-inference-api,{es} Inference API>>*.
Expand Down Expand Up @@ -137,4 +135,4 @@ POST _search
* Read the <<retriever,retriever reference documentation>> for syntax and implementation details
* Learn more about the <<retrievers-overview,retrievers>> abstraction
* Learn more about the Elastic <<inference-apis,Inference APIs>>
* Check out our https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/integrations/cohere/cohere-elasticsearch.ipynb[Python notebook] for using Cohere with {es}
* Check out our https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/integrations/cohere/cohere-elasticsearch.ipynb[Python notebook] for using Cohere with {es}
13 changes: 7 additions & 6 deletions docs/reference/search/retriever.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -536,13 +536,14 @@ Refer to <<semantic-reranking>> for a high level overview of semantic re-ranking

===== Prerequisites

To use `text_similarity_reranker` you must first set up an inference endpoint for the `rerank` task using the <<put-inference-api, Create {infer} API>>.
The endpoint should be set up with a machine learning model that can compute text similarity.
Refer to {ml-docs}/ml-nlp-model-ref.html#ml-nlp-model-ref-text-similarity[the Elastic NLP model reference] for a list of third-party text similarity models supported by {es}.
To use `text_similarity_reranker`, you can rely on the preconfigured `.rerank-v1-elasticsearch` inference endpoint, which uses the <<inference-example-elastic-reranker,Elastic Rerank model>> and serves as the default if no `inference_id` is provided.
This model is optimized for reranking based on text similarity. If you'd like to use a different model, you can set up a custom inference endpoint for the `rerank` task using the <<put-inference-api, Create {infer} API>>.
The endpoint should be configured with a machine learning model capable of computing text similarity.
Refer to {ml-docs}/ml-nlp-model-ref.html#ml-nlp-model-ref-text-similarity[the Elastic NLP model reference] for a list of third-party text similarity models supported by {{es}}.

You have the following options:

* Use the the built-in <<inference-example-elastic-reranker,Elastic Rerank>> cross-encoder model via the inference API's {es} service.
* Use the built-in Elastic Rerank cross-encoder model via the inference APIs {{es}} service. See <<inference-example-elastic-reranker, this example>> for creating an endpoint using the Elastic Rerank model.
* Use the <<infer-service-cohere,Cohere Rerank inference endpoint>> with the `rerank` task type.
* Use the <<infer-service-google-vertex-ai,Google Vertex AI inference endpoint>> with the `rerank` task type.
* Upload a model to {es} with {eland-docs}/machine-learning.html#ml-nlp-pytorch[Eland] using the `text_similarity` NLP task type.
Expand Down Expand Up @@ -582,9 +583,9 @@ The document field to be used for text similarity comparisons.
This field should contain the text that will be evaluated against the `inferenceText`.

`inference_id`::
(Required, `string`)
(Optional, `string`)
+
Unique identifier of the inference endpoint created using the {infer} API.
Unique identifier of the inference endpoint created using the {infer} API. If you don’t specify an inference endpoint, the `inference_id` field defaults to `.rerank-v1-elasticsearch`, a preconfigured endpoint for the elasticsearch `.rerank-v1` model.

`inference_text`::
(Required, `string`)
Expand Down