Greater introspection into the successful model when using FallbackModel

[dhimmel]

My initial use case here is to know which provider and model combination produced the output when using FallbackModel. Right now we can know the model name (possibly different than the input model name str) but not the provider. Example:

from pydantic_ai import Agent
from pydantic_ai.models import infer_model
from pydantic_ai.models.fallback import FallbackModel

agent = Agent(
    model=FallbackModel([
        infer_model("provider_1:model_1"),
        infer_model("provider_2:model_1"),
    ]),
)
result = await agent.run(
    user_prompt="This is a test. Respond with only the word 'okay'."
)
result.new_messages()[-1].model_name

But I think there are a broader set of related use cases:

• knowing which model succeeded when there are multiple instances of the same provider model pair in a Fallback scenario with different settings post FallbackModel to allow model_settings specific to each model #2119
• introspection on failed model calls prior to the success call

Is there a way for AgentRunResult to learn more about the models and calls behind it including each model in a Fallback scenario?

Related

• Fallback Model #516
• Add FallbackModel support #894
• FallbackModel to allow model_settings specific to each model #2119

[DouweM]

@dhimmel Thanks for the request, I think it makes sense to add a model field to AgentRunResult.

In this method that builds the AgentRunResult, we should be able to get the model from self._graph_run.deps.model:

pydantic-ai/pydantic_ai_slim/pydantic_ai/agent.py

Lines 1946 to 1962 in b31c77d

	@property
	def result(self) -> AgentRunResult[OutputDataT] | None:
	"""The final result of the run if it has ended, otherwise `None`.
	Once the run returns an [`End`][pydantic_graph.nodes.End] node, `result` is populated
	with an [`AgentRunResult`][pydantic_ai.agent.AgentRunResult].
	"""
	graph_run_result = self._graph_run.result
	if graph_run_result is None:
	return None
	return AgentRunResult(
	graph_run_result.output.output,
	graph_run_result.output.tool_name,
	graph_run_result.state,
	self._graph_run.deps.new_message_index,
	self._traceparent(required=False),
	)

Would you be up for submitting a PR that implements that?

[dhimmel]

I think it makes sense to add a model field to AgentRunResult

Okay so AgentRunResult.model would be a pydantic_ai.models.Model although not a pydantic_ai.models.fallback.FallbackModel because it would reference to the specific succeeding model.

Would you be up for submitting a PR that implements that?

Yes, but I am AFK until 2025-07-15, so there will be a slight delay getting started.