Skip to content

server: inject date_string in llama 3.x template + fix date for firefunction v2 #12802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
May 15, 2025

Conversation

ochafik
Copy link
Collaborator

@ochafik ochafik commented Apr 7, 2025

Fixes #12729

  • Now using the chat handler common_chat_params_init_llama_3_x & common_chat_params_init_functionary_v3_1_llama_3_1 even when there's no tools
  • Pass "now" time to Minja & use it to define date_string for llama 3.x
  • Added an e2e test (examples/server/tests/unit/test_template.py) to check the date appears in the prompt for llama 3.3 and firefunction v2

cc/ @CISC

@ochafik ochafik changed the title Inject date_string in llama 3.x + fix date for functionary v2 jinja: inject date_string in llama 3.x + fix date for functionary v2 Apr 7, 2025
@ochafik ochafik changed the title jinja: inject date_string in llama 3.x + fix date for functionary v2 jinja: inject date_string in llama 3.x + fix date for firefunction v2 Apr 7, 2025
@github-actions github-actions bot added examples python python script changes server labels Apr 7, 2025
@github-actions github-actions bot added the testing Everything test related label Apr 7, 2025
@ochafik ochafik marked this pull request as ready for review April 7, 2025 23:06
@ochafik ochafik requested a review from ngxson as a code owner April 7, 2025 23:06
@ochafik ochafik changed the title jinja: inject date_string in llama 3.x + fix date for firefunction v2 server: inject date_string in llama 3.x template + fix date for firefunction v2 Apr 7, 2025
@ggerganov
Copy link
Member

@ochafik Good to merge.

@CISC
Copy link
Collaborator

CISC commented May 14, 2025

@ochafik Not sure what's going on, could be related to changes in master?

FAILED unit/test_tool_call.py::test_completion_with_required_tool_tiny_fast[meta-llama-Llama-3.3-70B-Instruct-tool2-code] - AssertionError: Expected 1 tool call in {'role': 'assistant', 'content': '<|python_tag|>python.call(code=" her mommy. As she was done, she saw a big, scary. As she was done, she saw a big, scary. She was so happy and prouded all the peace. After, she couldn\'try again. Frace was so happy and prouded all the peach. Fraggagel and the peach. Fraggether, they were all the peach on their face. Once upon a time, there was a little girl named Lily. She loved to play outside in the park. One day, she went to the park with her mommy and daddy. They were very happy and had a big box. Lily was very happy and proud of herself for being so clean. When they got to the park, Lily saw a big box with a big box. She wanted to play with it, but she didn\'t want to go to the park. When she got to the park, she saw a big box. She was sad and didn\'t know what to do. As she was cleaning, she saw a big box. The boy was sad and didn\'t know what to do. Lily was sad and didn\'t know what to do. She didn\'t know what to do. After a while, she saw a big, scary box. The boy was sad and didn\'t know what to do. He didn\'t know what to do. He didn\'t know what to do. He didn\'t know w'}
assert (None)

@ochafik ochafik merged commit aa48e37 into ggml-org:master May 15, 2025
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
examples python python script changes server testing Everything test related tool calling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Eval bug: Jinja not replacing date_string
3 participants