Skip to content

Support Seed-Coder chat template #13472

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Support Seed-Coder chat template #13472

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions src/llama-chat.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ static const std::map<std::string, llm_chat_template> LLM_CHAT_TEMPLATES = {
{ "bailing", LLM_CHAT_TEMPLATE_BAILING },
{ "llama4", LLM_CHAT_TEMPLATE_LLAMA4 },
{ "smolvlm", LLM_CHAT_TEMPLATE_SMOLVLM },
{ "seed-coder", LLM_CHAT_TEMPLATE_SEED_CODER },
};

llm_chat_template llm_chat_template_from_str(const std::string & name) {
Expand Down Expand Up @@ -183,6 +184,9 @@ llm_chat_template llm_chat_detect_template(const std::string & tmpl) {
return LLM_CHAT_TEMPLATE_BAILING;
} else if (tmpl_contains("<|header_start|>") && tmpl_contains("<|header_end|>")) {
return LLM_CHAT_TEMPLATE_LLAMA4;
} else if (tmpl_contains("raise_exception") && tmpl_contains("System role not supported") &&
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will clash with many other chat templates, many of them have the same messages like this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this only applies to the Reasoning model, see Instruct:
https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Instruct/blob/main/tokenizer_config.json#L1029

In fact, these models don't seem to have proper chat templates at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 I actually compared the templates from llama4 and ling-lite before choosing these conditions. Do you have any recommended strings to choose? Thanks!

{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% set role = message['role'] %}{{ bos_token + role + '\n' + message['content'] | trim + eos_token }}{% endfor %}{% if add_generation_prompt %}{{ bos_token + 'assistant\n'}}{% endif %}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this only applies to the Reasoning model, see Instruct: https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Instruct/blob/main/tokenizer_config.json#L1029

In fact, these models don't seem to have proper chat templates at all?

Ah... I didn’t expect that.

Copy link
Collaborator

@CISC CISC May 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a very strange chat format, and as long as the chat templates are so sparse I don't think you can expect to detect them properly, use --jinja for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to @shenz2-2000’s reply in ByteDance-Seed/Seed-Coder#2, the difference in chat template configurations between the instruct and reasoning models is expected.

The only shared substring I could find is:
'\n' + message['content'] | trim + eos_token }}{% endfor %}{% if add_generation_prompt %}{{ bos_token + 'assistant\n'}}{% endif %}, which also appears quite generic. So currently, I’m unsure how to reliably identify the model type based on the template alone.

tmpl_contains("Conversation roles must alternate user/assistant/user/assistant/...")) {
return LLM_CHAT_TEMPLATE_SEED_CODER;
}
return LLM_CHAT_TEMPLATE_UNKNOWN;
}
Expand Down Expand Up @@ -643,6 +647,17 @@ int32_t llm_chat_apply_template(
if (add_ass) {
ss << "Assistant:";
}
} else if (tmpl == LLM_CHAT_TEMPLATE_SEED_CODER) {
// Seed-Coder
for (auto message : chat) {
std::string role(message->role);
if (role == "user") {
ss << "<[begin▁of▁sentence]>" << role << "\n" << trim(message->content) << "<[end▁of▁sentence]>";
}
}
if (add_ass) {
ss << "<[begin▁of▁sentence]>assistant\n";
}
} else {
// template not supported
return -1;
Expand Down
1 change: 1 addition & 0 deletions src/llama-chat.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ enum llm_chat_template {
LLM_CHAT_TEMPLATE_BAILING,
LLM_CHAT_TEMPLATE_LLAMA4,
LLM_CHAT_TEMPLATE_SMOLVLM,
LLM_CHAT_TEMPLATE_SEED_CODER,
LLM_CHAT_TEMPLATE_UNKNOWN,
};

Expand Down
Loading