[mrope][Qwen2-VL] Fix edge case where getting index of image/video token can potentially throw in default vl mrope implementation. #23895
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
In #23838, Gemini caught an edge case where checking image/video token in
input_tokens
does not guaranteeinput_tokens.index(image_token_id, st)
won't throw, hence the missing try-catch block is needed. After discussion with @DarkLight1337 we want to add this check in the default vl mrope code as well.Test Plan
Note that there's no meaningful perf impact since checking an item in the list is generally very fast:
e2e benchmarks (reference only):
Without this PR:
With this PR:
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.md
andexamples
for a new model.