[Fix] correct tool_id for kimi-k2 when use tool_choice=required #21259

MoyanZitto · 2025-07-20T18:49:33Z

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

This is to fix tool_id for kimi-k2 when use tool_choice=required. Now it will return tool id with format functions.func_name:idx instead of random string.

Test Plan

Additional tests were added to test_completion_with_function_calling, and corresponding changes to the model_type hack were also required.

Test Result

tests passed.

(Optional) Documentation Update

github-actions · 2025-07-20T18:49:44Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

gemini-code-assist

Code Review

This pull request implements deterministic tool IDs for the kimi-k2 model. The changes look good, but there's a critical bug in the non-streaming generation path when multiple choices (n > 1) are requested, which could lead to non-unique tool call IDs. There's also a potential TypeError in the streaming path that should be addressed to improve code robustness.

vllm/entrypoints/openai/serving_chat.py

simon-mo · 2025-07-22T20:38:09Z

Thank you for the fix, can you please address some of the comments? Thank you!

MoyanZitto · 2025-07-23T06:34:50Z

Thank you for the fix, can you please address some of the comments? Thank you!

yeah I'll fix them as soon as possible. Orz

mergify · 2025-07-24T03:50:33Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @MoyanZitto.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

mergify · 2025-07-25T09:31:15Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @MoyanZitto.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

vllm/entrypoints/openai/serving_chat.py

vllm/entrypoints/chat_utils.py

aarnphm

I don't think we have to make this change to interface just for kimi. Let's just make a special case for kimi for now.

vllm/entrypoints/chat_utils.py

aarnphm · 2025-07-29T16:45:25Z

vllm/entrypoints/openai/protocol.py

This wouldn't work, given that the default would always be random, so not sure how this would work with kimi here.

kimi-k2 will always set ToolCall.id manually, so the random default value doesn't matter.

vllm/entrypoints/chat_utils.py

MoyanZitto · 2025-08-18T18:31:46Z

@simon-mo 3 hours ago, openai v1.100.0 removed ResponseTextConfig from openai.types.responses, see: https://github.com/openai/openai-python/blob/adb1af8073391a6d58be9c13cfa0664c04d859e2/src/openai/types/responses/response.py

which cause readthedocs build failed

addressed

…-project#21259) Co-authored-by: wangzhengtao <[email protected]> Signed-off-by: Duncan Moss <[email protected]>

…-project#21259) Co-authored-by: wangzhengtao <[email protected]>

…-project#21259) Co-authored-by: wangzhengtao <[email protected]> Signed-off-by: Xiao Yu <[email protected]>

…-project#21259) Co-authored-by: wangzhengtao <[email protected]>

MoyanZitto requested a review from aarnphm as a code owner July 20, 2025 18:49

mergify bot added the frontend label Jul 20, 2025

gemini-code-assist bot reviewed Jul 20, 2025

View reviewed changes

MoyanZitto force-pushed the main branch from 54976c3 to 2180294 Compare July 23, 2025 18:03

mergify bot added the needs-rebase label Jul 24, 2025

MoyanZitto closed this Jul 25, 2025

MoyanZitto force-pushed the main branch from 2180294 to 40d86ee Compare July 25, 2025 09:10

correct tool_id for kimi-k2 when use tool_choice=required

238687c

MoyanZitto reopened this Jul 25, 2025

mergify bot removed the needs-rebase label Jul 25, 2025

wangzhengtao added 4 commits July 25, 2025 10:52

fix lint..

bf25f0b

make lint happy

f21b062

fix reasoning content

56e467a

add tests

e2e31a7

MoyanZitto requested review from DarkLight1337, robertgshaw2-redhat and simon-mo as code owners July 29, 2025 09:01

DarkLight1337 reviewed Jul 29, 2025

View reviewed changes

vllm/entrypoints/openai/serving_chat.py Outdated Show resolved Hide resolved

mergify bot added deepseek Related to DeepSeek models llama Related to Llama models tool-calling labels Jul 29, 2025

github-project-automation bot added this to Tool Calling Jul 29, 2025

chaunceyjiang reviewed Jul 29, 2025

View reviewed changes

vllm/entrypoints/openai/serving_chat.py Outdated Show resolved Hide resolved

chaunceyjiang reviewed Jul 29, 2025

View reviewed changes

vllm/entrypoints/chat_utils.py Outdated Show resolved Hide resolved

aarnphm previously requested changes Jul 29, 2025

View reviewed changes

wangzhengtao and others added 4 commits August 18, 2025 13:37

fix

ce25e8d

let's make mypy happy

de21733

Merge branch 'main' into main

f2c6442

make mypy happy

b9a7098

Merge branch 'main' into main

1b1c61f

simon-mo enabled auto-merge (squash) August 20, 2025 06:17

github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 20, 2025

aarnphm approved these changes Aug 20, 2025

View reviewed changes

fix magic mock

bd40b89

auto-merge was automatically disabled August 20, 2025 11:38
Head branch was pushed to by a user without write access

simon-mo merged commit 582bbe6 into vllm-project:main Aug 20, 2025
38 checks passed

djmmoss pushed a commit to djmmoss/vllm that referenced this pull request Aug 21, 2025

[Fix] correct tool_id for kimi-k2 when use tool_choice=required (vllm…

c68cadb

…-project#21259) Co-authored-by: wangzhengtao <[email protected]> Signed-off-by: Duncan Moss <[email protected]>

epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 28, 2025

[Fix] correct tool_id for kimi-k2 when use tool_choice=required (vllm…

e43f00f

…-project#21259) Co-authored-by: wangzhengtao <[email protected]>

xiao-llm pushed a commit to xiao-llm/vllm that referenced this pull request Aug 28, 2025

[Fix] correct tool_id for kimi-k2 when use tool_choice=required (vllm…

c213c42

…-project#21259) Co-authored-by: wangzhengtao <[email protected]> Signed-off-by: Xiao Yu <[email protected]>

zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Aug 28, 2025

[Fix] correct tool_id for kimi-k2 when use tool_choice=required (vllm…

83dbb46

…-project#21259) Co-authored-by: wangzhengtao <[email protected]>

mengxingkongzhouhan pushed a commit to mengxingkongzhouhan/vllm that referenced this pull request Aug 30, 2025

[Fix] correct tool_id for kimi-k2 when use tool_choice=required (vllm…

d8ac073

…-project#21259) Co-authored-by: wangzhengtao <[email protected]>

zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Sep 3, 2025

[Fix] correct tool_id for kimi-k2 when use tool_choice=required (vllm…

e44cf4d

…-project#21259) Co-authored-by: wangzhengtao <[email protected]>

eraser00 mentioned this pull request Sep 18, 2025

[Feature] Enforce the tool_call.index increase with tool calls history count when using kimi-k2 parser sgl-project/sglang#10600

Closed

2 tasks

FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025

[Fix] correct tool_id for kimi-k2 when use tool_choice=required (vllm…

1b6a7f2

…-project#21259) Co-authored-by: wangzhengtao <[email protected]>

Uh oh!

[Fix] correct tool_id for kimi-k2 when use tool_choice=required #21259

[Fix] correct tool_id for kimi-k2 when use tool_choice=required #21259

Uh oh!

Conversation

MoyanZitto commented Jul 20, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Essential Elements of an Effective PR Description Checklist

Purpose

Test Plan

Test Result

(Optional) Documentation Update

Uh oh!

github-actions bot commented Jul 20, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

simon-mo commented Jul 22, 2025

Uh oh!

MoyanZitto commented Jul 23, 2025

Uh oh!

mergify bot commented Jul 24, 2025

Uh oh!

mergify bot commented Jul 25, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aarnphm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aarnphm Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

MoyanZitto Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MoyanZitto commented Aug 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

MoyanZitto commented Jul 20, 2025 •

edited by github-actions bot

Loading