Eval bug: Mistral-Small not working on vulkan

[daniandtheweb]

Name and Version

version: 5835 (6491d6e)
built with cc (GCC) 15.1.1 20250425 for x86_64-pc-linux-gnu

Operating systems

Linux

GGML backends

Vulkan

Hardware

Ryzen 7 9700X + Radeon 7800XT

Models

Mistral-Small IQ4_XS

Problem description & steps to reproduce

Mistral-Small doesn't work on vulkan, the first tokens after a prompt get generated correctly but after around 200 tokens the generation speed drops substantially and the output becomes incoherent.

First Bad Commit

This issue has been first introduced in commit 8875523.

Relevant log output

vk_amdvlk ./llama-server -t 8 -fa -ctk q8_0 -ctv q8_0 -c 32768 -m ~/Applications/chat/gguf/Mistral-Small-3.2-24B-Instruct-2506-IQ4_XS.gguf -ngl 100

[CISC]

Most likely fixed in #14545 try b5834.

[daniandtheweb]

The issuse has been tested on the latest commit (I mistakenly pasted the old version name in the issue).

[jeffbolznv]

I haven't been able to reproduce this. I'm using llama-cli, commit 6491d6e, on RTX 5090 with this command line: llama-cli -t 8 -fa -ctk q8_0 -ctv q8_0 -c 32768 -m C:\models\Mistral-Small-3.2-24B-Instruct-2506-IQ4_XS.gguf -ngl 100 -p "please write a long story"

[daniandtheweb]

I've just retested it with the radv driver instead of AMD's amdvlk and it works. The issue seems driver specific.

[daniandtheweb]

Given the similarity that amdvlk and the windows driver have I have just tested the latest build on windows and the same issue seems to be happening there.

[netrunnereve]

I'm not seeing this with the Q4_K_S Mistral Small model, which I use regularly. Adding your flags -fa -ctk q8_0 -ctv q8_0 -c 32768 didn't make a difference.

I run it with RADV and no coopmat, so maybe try that and see if it works?

[netrunnereve]

LOL Github only showed your reply after I posted, so ignore what I just said. The AMD drivers are probably acting up again.