Eval bug: Qwen3, failed to parse chat template (jinja)

[matteoserva]

Name and Version

ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1103 (0x1103), VMM: no, Wave Size: 32
version: 5217 (e98b369)
built with cc (Debian 12.2.0-14) 12.2.0 for x86_64-linux-gnu

Operating systems

Linux

GGML backends

HIP

Hardware

amd 780M, gfx1103

Models

Qwen 3 models: Qwen_Qwen3-30B-A3B-Q5_K_L.gguf

Problem description & steps to reproduce

llama.cpp fails to parse the Qwen3 chat template, the error is related to the list slicing syntax

First Bad Commit

No response

Relevant log output

common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
common_chat_templates_init: failed to parse chat template (defaulting to chatml): Expected value expression at row 18, column 30:
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
                             ^
    {%- set index = (messages|length - 1) - loop.index0 %}

srv          init: initializing slots, n_slots = 1

[CISC]

Yep, minja doesn't support the step slicing parameter.

[CISC]

Probably not worth fully supporting step, but rather just special-casing [::-1].

[CISC]

The str startswith method is also not supported, seems like an oversight as endswith is.

[shivak]

As a temporary workaround, the following template implements the same logic without reverse stepping, startswith or endswith. To use, llama-server --jinja --chat-template-file qwen3-workaround.jinja -m .... For me, this morning, it's working with/without tools, but please double check.

{%- if tools %}
    {{- '<|im_start|>system\n' }}
    {%- if messages[0].role == 'system' %}
        {{- messages[0].content + '\n\n' }}
    {%- endif %}
    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
    {%- if messages[0].role == 'system' %}
        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}

{%- set ns = namespace(multi_step_tool=true, last_query_index=0) %}
{%- for message in messages %}
    {%- if ns.multi_step_tool %}
        {%- if message.role == "user" %}
            {%- if not (message.content[0:14] == '<tool_response>' and message.content[-15:] == '</tool_response>') %}
                {%- set ns.multi_step_tool = false %}
                {%- set ns.last_query_index = loop.index0 %}
            {%- endif %}
        {%- endif %}
    {%- endif %}
{%- endfor %}

{%- for message in messages %}
    {%- if message.role == "user" or (message.role == "system" and loop.index0 != 0) %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
    {%- elif message.role == "assistant" %}
        {%- set content = message.content %}
        {%- set reasoning_content = '' %}
        {%- if message.reasoning_content is defined and message.reasoning_content %}
            {%- set reasoning_content = message.reasoning_content %}
        {%- else %}
            {%- if '<think>' in message.content and '</think>' in message.content %}
                {%- set think_start = message.content.find('<think>') + 7 %}
                {%- set think_end = message.content.find('</think>') %}
                {%- set reasoning_content = message.content[think_start:think_end] %}
                {%- set content = message.content[think_end + 8:] %}
            {%- endif %}
        {%- endif %}
        {%- if loop.index0 > ns.last_query_index %}
            {%- if loop.last or (not loop.last and reasoning_content) %}
                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
            {%- else %}
                {{- '<|im_start|>' + message.role + '\n' + content }}
            {%- endif %}
        {%- else %}
            {{- '<|im_start|>' + message.role + '\n' + content }}
        {%- endif %}
        {%- if message.tool_calls %}
            {%- for tool_call in message.tool_calls %}
                {{- '\n' }}
                {%- if tool_call.function %}
                    {%- set tool_call = tool_call.function %}
                {%- endif %}
                {{- '<tool_call>\n{"name": "' + tool_call.name + '", "arguments": ' }}
                {%- if tool_call.arguments is string %}
                    {{- tool_call.arguments }}
                {%- else %}
                    {{- tool_call.arguments | tojson }}
                {%- endif %}
                {{- '}\n</tool_call>' }}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- set prev_role = (loop.index0 > 0) and (messages[loop.index0 - 1].role) or '' %}
        {%- if loop.index0 == 0 or prev_role != "tool" %}
            {{- '<|im_start|>user\n' }}
        {%- endif %}
        {{- '<tool_response>\n' + message.content + '\n</tool_response>' }}
        {%- set next_role = (loop.index0 + 1 < messages|length) and (messages[loop.index0 + 1].role) or '' %}
        {%- if loop.index0 == messages|length - 1 or next_role != "tool" %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}

{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking == false %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- endif %}
{%- endif %}

Since passing enable_thinking also isn't supported yet, you might want to eliminate the associated conditional at the end. For example, to disable thinking:

{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {{- '<think>\n\n</think>\n\n' }}
{%- endif %}

[x4080]

ahh thats why it wont work, I thought that I make a mistake or something

I got this error using openai api

openai.InternalServerError: Error code: 500 - {'error': {'code': 500, 'message': "Cannot subscript null at row 23, column 40:\n    {%- set tool_start_length = tool_start|length %}\n    {%- set start_of_message = message.content[:tool_start_length] %}\n                                       ^\n    {%- set tool_end = '</tool_response>' %}\n at row 23, column 5:\n    {%- set tool_start_length = tool_start|length %}\n    {%- set start_of_message = message.content[:tool_start_length] %}\n    ^\n    {%- set tool_end = '</tool_response>' %}\n at row 18, column 39:\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for forward_message in messages %}\n                                      ^\n    {%- set index = (messages|length - 1) - loop.index0 %}\n at row 18, column 1:\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for forward_message in messages %}\n^\n    {%- set index = (messages|length - 1) - loop.index0 %}\n at row 1, column 1:\n{%- if tools %}\n^\n    {{- '<|im_start|>system\\n' }}\n", 'type': 'server_error'}}