Skip to content

ES|QL dense vector support #125783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
23 tasks
carlosdelest opened this issue Mar 27, 2025 · 1 comment
Open
23 tasks

ES|QL dense vector support #125783

carlosdelest opened this issue Mar 27, 2025 · 1 comment
Assignees
@carlosdelest
Labels
:Analytics/ES|QL AKA ESQL >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@carlosdelest
Copy link
Member

carlosdelest commented Mar 27, 2025 •
edited
Loading

Description

Provide support for dense_vector search in ES|QL, including:

dense_vector field support:

  • field_caps support to retrieve element type for dense_vector field types
  • float_vector element type using Float internally instead of Double
  • byte_vector element type
  • bit_vector element type
  • Casting operations: TO_FLOAT_VECTOR, TO_BYTE_VECTOR, TO_BIT_VECTOR
  • Implicit casting
    • Multivalues to float vectors
    • Strings (hex strings) to byte / bit vectors

ES|QL similarity functions support

  • V_SIMILARITY operation - delegates into appropriate similarity function.
  • V_COSINE_SIMILARITY – calculates cosine similarity
  • V_DOT_PRODUCT – calculates dot product
  • V_L1_NORM – calculates L1 distance
  • V_HAMMING – calculates Hamming distance
  • V_L2_NORM - calculates L2 distance
  • V_MAGNITUDE – returns a vector’s magnitude

Post-MVP - ES|QL operators support

  • CASE and COALESCE support
  • Support binary comparison (==)
  • Support COUNT and COUNT_DISTINCT
  • Add (scalar and vector version)
  • Subtract (scalar and vector version)
  • Multiply (scalar and vector version)
  • Divide (scalar and vector version)
  • Unary negation

Search support for dense_vector will be addressed in #126710
@carlosdelest carlosdelest added >enhancement needs:triage Requires assignment of a team area label labels Mar 27, 2025
@carlosdelest carlosdelest self-assigned this Mar 27, 2025
@carlosdelest carlosdelest mentioned this issue Mar 27, 2025
Open
@gbanasiak gbanasiak added the :Analytics/ES|QL AKA ESQL label Mar 28, 2025
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Mar 28, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Mar 28, 2025
@carlosdelest carlosdelest changed the title ES|QL dense vector search support ES|QL dense vector support Apr 11, 2025
@carlosdelest carlosdelest mentioned this issue Apr 11, 2025
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@carlosdelest carlosdelest

Labels
:Analytics/ES|QL AKA ESQL >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@carlosdelest @gbanasiak @elasticsearchmachine