feat: Function calling with output schema enforcement

[jeffreyftang]

Introduces function-calling capabilities with JSON schema enforcement on output.

Example using Mistral-7B-Instruct-v0.3:

resp = client.chat.completions.create(
    model="",  # optional: specify an adapter ID here
    messages=[
        {
            "role": "user",
            "content": "What is the weather like in New York?",
        },
    ],
    max_tokens=512,
    tools= [
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        },
                        "format": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                            "description": "The temperature unit to use. Infer this from the users location."
                        }
                    },
                    "required": ["location", "format"]
                }
            }
        }
    ],
    tool_choice="auto",
)

print(resp)


ChatCompletion(id='null', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='0', function=Function(arguments={'format': 'fahrenheit', 'location': 'New York, NY'}, name='get_current_weather', description=None), type='function')]))], created=1729121416, model='Qwen/Qwen2.5-32B-Instruct', object='text_completion', service_tier=None, system_fingerprint='0.1.0-native', usage=CompletionUsage(completion_tokens=28, prompt_tokens=249, total_tokens=277, completion_tokens_details=None, prompt_tokens_details=None))

[llama-shepard]

@tgaddair @magdyksaleh

Waiting for this merge to try out Function Calling in Lorax

[jeffreyftang]

Failing test doesn't seem related, but will have to dig more.

[llama-shepard]

@magdyksaleh Still waiting for this feature. I want to deploy Mistral Nemo and Llama 3.1 models with function calling

[llama-shepard]

I have been waiting for function calling feature for long time and got exited for this PR. But this PR is still not merged.

@tgaddair @jeffreyftang

[jeffreyftang]

@llama-shepard Sorry, I've been preoccupied with other obligations. What's remaining on this PR is to verify that pulling in chat templates will provide support for models other than mistral 3 (which is already working).

[nikhilh]

This would be extremely useful for me. When can we expect this to be merged?

[prd-tuong-nguyen]

@tgaddair Hey, do you have any documents regarding this feature?