Description
Initial Checks
- I confirm that I'm using the latest version of Pydantic AI
- I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue
Description
Function call fails on some LLM when using nested classes as output_type
It fails with:
Qwen/Qwen2.5-72B-Instruct-Turbo
on together.aiQwen/Qwen3-4B-fast
on nebius.comQwen/Qwen3-235B-A22B-fp8-tput
on together.ai
It succeeds with:
meta-llama/Llama-3.3-70B-Instruct-Turbo
on together.aiQwen3-4B-Q6_K.gguf
local on llama.cpp
To replicate the bug, see the example code:
When using the classes Count
, Size
, Name
as output_type
, all models succeed, when using NameAndCount
some models fails. The error messages are the following:
for togethe.ai models pydantic_ai.exceptions.ModelHTTPError: status_code: 400, model_name: Qwen/Qwen2.5-72B-Instruct-Turbo, body: {'message': 'invalid tools grammar: Aborted(). Build with -sASSERTIONS for more info.', 'type': 'invalid_request_error', 'param': 'tools', 'code': None}
for the nebius.com model: pydantic_ai.exceptions.ModelHTTPError: status_code: 400, model_name: Qwen/Qwen3-4B-fast, body: {'detail': "Invalid request. Please check the parameters and try again. Details: 1 validation error for list[function-wrap[__log_extra_fields__()]]\n Invalid JSON: EOF while parsing a value at line 1 column 0 [type=json_invalid, input_value='', input_type=str]
Possibly related to #1561 and #1414
Example Code
import os
from pydantic import BaseModel, Field
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.openai import OpenAIProvider
from pydantic_ai.settings import ModelSettings
class Count(BaseModel):
count: int = Field(
description="count",
)
class Size(BaseModel):
size: float = Field(
description="size",
)
class Name(BaseModel):
name: str = Field(
description="name",
)
class NameAndCount(BaseModel):
name: str = Field(
description="name",
)
count: Count
# fails:
# pydantic_ai.exceptions.ModelHTTPError: status_code: 400, model_name: Qwen/Qwen3-235B-A22B-fp8-tput, body: {'message': 'invalid tools grammar: Aborted(). Build with -sASSERTIONS for more info.', 'type': 'invalid_request_error', 'param': 'tools', 'code': None}
provider = OpenAIModel(
model_name="Qwen/Qwen3-235B-A22B-fp8-tput",
provider=OpenAIProvider(
base_url="https://api.together.xyz/v1",
api_key=os.getenv("TOGETHER_API_KEY"),
))
# succeeds
# provider = OpenAIModel(
# model_name="meta-llama/Llama-3.3-70B-Instruct-Turbo",
# provider=OpenAIProvider(
# base_url="https://api.together.xyz/v1",
# api_key=os.getenv("TOGETHER_API_KEY"),
# ))
# fails:
# pydantic_ai.exceptions.ModelHTTPError: status_code: 400, model_name: Qwen/Qwen2.5-72B-Instruct-Turbo, body: {'message': 'invalid tools grammar: Aborted(). Build with -sASSERTIONS for more info.', 'type': 'invalid_request_error', 'param': 'tools', 'code': None}
# provider = OpenAIModel(
# model_name="Qwen/Qwen2.5-72B-Instruct-Turbo",
# provider=OpenAIProvider(
# base_url="https://api.together.xyz/v1",
# api_key=os.getenv("TOGETHER_API_KEY"),
# ))
# fails:
# pydantic_ai.exceptions.ModelHTTPError: status_code: 400, model_name: Qwen/Qwen3-4B-fast, body: {'detail': "Invalid request. Please check the parameters and try again. Details: 1 validation error for list[function-wrap[__log_extra_fields__()]]\n Invalid JSON: EOF while parsing a value at line 1 column 0 [type=json_invalid, input_value='', input_type=str]\n For further information visit https://errors.pydantic.dev/2.11/v/json_invalid"}
# provider = OpenAIModel(
# model_name="Qwen/Qwen3-4B-fast",
# provider=OpenAIProvider(
# base_url="https://api.studio.nebius.com/v1",
# api_key=os.getenv("NEBIUS_API_KEY"),
# ),
# )
# succeeds (Qwen3-4B-Q6_K.gguf on llama.cpp)
# provider = OpenAIModel(
# model_name="Qwen3-4B-Q6_K.gguf",
# provider=OpenAIProvider(
# base_url="http://localhost:8080/v1",
# api_key="llamacpp",
# ),
# )
model_settings = ModelSettings(
max_tokens=2048,
temperature=0.0,
timeout=60,
)
agent = Agent(
provider,
output_type=Size, # using Name, Count, Size works fine, but using NameAndCount fails
model_settings=model_settings,
retries=2,
instrument=True,
system_prompt="""
#Instructions:
- Extract the count, the name and the size from the user input.""",
)
async def main():
import logfire
from loguru import logger
logfire.configure(
environment="test",
service_name="bug.py",
console=logfire.ConsoleOptions(verbose=True),
send_to_logfire=True,
)
logger.configure(handlers=[logfire.loguru_handler()])
logfire.instrument_httpx(capture_all=True)
user_input = """
<UserInput>
The count is 9, the name is 'test', the size is 55
</UserInput>
"""
result = await agent.run(user_input)
print(result.output)
if __name__ == "__main__":
import asyncio
asyncio.run(main())
Python, Pydantic AI & LLM client version
Python 3.12.8 (main, Jan 14 2025, 23:36:58) [Clang 19.1.6 ] on darwin
Pydantic-ai 0.1.10
Qwen/Qwen3-235B-A22B-fp8-tput
meta-llama/Llama-3.3-70B-Instruct-Turbo
Qwen/Qwen2.5-72B-Instruct-Turbo
Qwen/Qwen3-4B-fast
Qwen3-4B-Q6_K.gguf