Leverage optimized native float32 vector scorers. #130541
Open
+1,521
−25
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ChrisHegarty
added
:Search Relevance/Vectors
ChrisHegarty
added
test-windows
|
The micro benchmarks all show approx 2x performance improvement in scorer operations, all platforms. For example: Apple Mac M2, AArch64 Scorer benchmark. Compare |
ChrisHegarty
added
the
cloud-deploy
ldematte
reviewed
Jul 4, 2025
There was a problem hiding this comment.
I've concentrated mainly on the native (C) part and its interaction (NativeAccess/VectorLibrary), and that looks good to me.
I have looked at the benchmarks and tests and they look sensible, but I think it's better to have another pair of eyes on the other parts (VectorScorer/lucene interaction)
|
Benchmarks looks good, and they are a testament of the goodness of Panama and its usage in Lucene! Hopefully OpenJDK will get rid of the bug "soon" :) |
ChrisHegarty
added a commit
that referenced
this pull request
Jul 4, 2025
…ions (#130635) This commit adds low-level optimized Neon, AVX2, and AVX 512 float32 vector operations; cosine, dot product, and square distance. The changes in this PR give approximately 2x performance increase for float32 vector operations across Linux/ Mac AArch64 and Linux x64 (both AVX2 and AVX 512). The performance increase comes mostly from being able to score the vectors off-heap (rather than copying on-heap before scoring). The low-level native scorer implementations show only approx ~3-5% improvement over the existing Panama Vector implementation. However, the native scorers allow to score off-heap. The use of Panama Vector with MemorySegments runs into a performance bug in Hotspot, where the bound is not optimally hoisted out of the hot loop (has been reported and acknowledged by OpenJDK) . This vector ops will be used by higher-level vector scorers in #130541
ChrisHegarty
changed the title
ChrisHegarty
changed the title
|
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
ChrisHegarty
commented
Jul 7, 2025
|
|
||
| // Minimal copy of Lucene99HnswVectorsFormat in order to provide an optimized scorer, | ||
| // which returns identical scores to that of the default flat vector scorer. | ||
| public class ES819HnswVectorsFormat extends KnnVectorsFormat { |
There was a problem hiding this comment.
I'm considering whether or not it is worth back porting this to 8.19.x.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Leverage optimized native float32 vector scorers.
The changes in this PR give approximately 2x performance increase for float32 vector operations across Linux/ Mac AArch64 and Linux x64 (both AVX2 and AVX 512).
The vector scorers leverage the native vector operations added by #130635.
The tests verify that the native scorers return similar values to that of the lucene scorers.
TODO: feature flag the new format so that we only create indices with that format when enabled.