Misc. bug: vulkan: performance regression after fd123cfead49eb32e386e26b8ef7a6d41554dda5

[rhjdvsgsgks]

Name and Version

fd123cf

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

vulkan backend

Command line

Problem description & steps to reproduce

model	size	params	backend	ngl	test	t/s
gemma3 12B Q5_K - Medium	8.09 GiB	11.77 B	Vulkan	99	pp512	61.69 ± 0.04
gemma3 12B Q5_K - Medium	8.09 GiB	11.77 B	Vulkan	99	tg128	21.87 ± 0.01

build: a53f7f7 (4908)

model	size	params	backend	ngl	test	t/s
gemma3 12B Q5_K - Medium	8.09 GiB	11.77 B	Vulkan	99	pp512	59.69 ± 0.05
gemma3 12B Q5_K - Medium	8.09 GiB	11.77 B	Vulkan	99	tg128	21.00 ± 0.25

build: fd123cf (4909)

First Bad Commit

fd123cf

Relevant log output

[0cc4m]

What GPU and OS are you using?

This is a little bit of a damned if you do, damned if you don't situation. Defaulting to smaller allocations fixes a number of OOM crashes due to fragmentation or driver problems, many of which have been reported over the last year. If it reduces performance slightly, that's regrettable, but I can't just go back to the old behaviour.

Please check if setting GGML_VK_SUBALLOCATION_BLOCK_SIZE=2147483648 fixes your performance regression already, otherwise you can go back to the old behaviour and performance by setting GGML_VK_SUBALLOCATION_BLOCK_SIZE=4294967296.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.