Skip to content

convert : converting mmproj for Qwen2/2.5VL from convert_hf_to_gguf #13209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
May 2, 2025
Merged

convert : converting mmproj for Qwen2/2.5VL from convert_hf_to_gguf #13209

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
792387b
wip
ngxson Apr 30, 2025
f7260c2
qwen2.5vl ok
ngxson Apr 30, 2025
f48f51d
Merge branch 'master' into xsn/convert_gguf_qwen2vl
ngxson Apr 30, 2025
b5e72ed
vision: fix models missing "text_config"
ngxson Apr 30, 2025
4fac7d4
add test
ngxson Apr 30, 2025
474933e
fix test repo name
ngxson Apr 30, 2025
651752f
fix 32B model
ngxson Apr 30, 2025
d96ef53
Revert "fix 32B model"
ngxson Apr 30, 2025
13e4ccc
clarify about 32B
ngxson Apr 30, 2025
6e31ddc
rm qwen surgery script
ngxson Apr 30, 2025
ef0bc7a
update llava/readme
ngxson Apr 30, 2025
62b7b1e
Merge branch 'master' into xsn/convert_gguf_qwen2vl
ngxson May 2, 2025
c030984
move V_ENC_EMBD_PATCH handling to Qwen2VLVisionModel
ngxson May 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
clarify about 32B
  • Loading branch information
@ngxson
ngxson committed Apr 30, 2025
commit 13e4cccb499bdecc8a173d58bd5ee5648ae5a8d2
2 changes: 1 addition & 1 deletion examples/llava/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ llama-mtmd-cli -hf ggml-org/Qwen2-VL-7B-Instruct-GGUF
# Qwen 2.5 VL
llama-mtmd-cli -hf ggml-org/Qwen2.5-VL-3B-Instruct-GGUF
llama-mtmd-cli -hf ggml-org/Qwen2.5-VL-7B-Instruct-GGUF
llama-mtmd-cli -hf ggml-org/Qwen2.5-VL-32B-Instruct-GGUF
llama-mtmd-cli -hf ggml-org/Qwen2.5-VL-72B-Instruct-GGUF
# NOTE: Qwen2.5-VL-32B text-only model is currently unusable
```

## How it works and what is `mmproj`?
Expand Down
2 changes: 1 addition & 1 deletion examples/llava/tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ if [ "$RUN_BIG_TESTS" = true ]; then
add_test "llama-mtmd-cli" "ggml-org/Qwen2-VL-7B-Instruct-GGUF:Q4_K_M"
add_test "llama-mtmd-cli" "ggml-org/Qwen2.5-VL-3B-Instruct-GGUF:Q4_K_M"
add_test "llama-mtmd-cli" "ggml-org/Qwen2.5-VL-7B-Instruct-GGUF:Q4_K_M"
add_test "llama-mtmd-cli" "ggml-org/Qwen2.5-VL-32B-Instruct-GGUF:Q4_K_M"
# add_test "llama-mtmd-cli" "ggml-org/Qwen2.5-VL-32B-Instruct-GGUF:Q4_K_M" # text model is broken, not sure why
# add_test "llama-mtmd-cli" "ggml-org/Qwen2.5-VL-72B-Instruct-GGUF:Q4_K_M" # too big
fi

Expand Down
Loading