Evaluation `Invocation.user_content` ValidationError when session has invocations without user events

[lifeodyssey]

Describe the bug

When adding a session to an eval set via the ADK FastAPI web server, the server fails with a Pydantic ValidationError on
Invocation.user_content.

The failure occurs when the stored Session contains at least one invocation_id whose events do not include any user-
authored event (author == "user"). In that case, EvaluationGenerator.convert_events_to_eval_invocations leaves user_content
as an empty string (""), but Invocation.user_content is typed as google.genai.types.Content, which causes validation to fail.

The error is triggered when calling the eval API:

POST /apps/{app_name}/eval-sets/{eval_set_id}/add-session

To Reproduce

Please share a minimal code and data to reproduce your problem.

1. Install the following in a new virtualenv:

   pip install google-adk==1.19.0 google-genai==1.52.0 fastapi==0.118.3 uvicorn==0.38.0 pydantic==2.12.4

2. Start the ADK web server for any app that persists sessions (e.g. using SQLite under .adk/session.db):

   adk web --agents-root ./adk_agents
   # or an equivalent entrypoint that uses AdkWebServer

3. Interact with the app to create a session such that for at least one invocation_id, all events are authored by agents/tools/
   models and no event has author == "user". (This can happen for internal/system-driven invocations that still share a valid
   invocation_id and are saved to the session.)

4. Use the eval endpoint or the web UI to add this session to an eval set:

   POST /apps/{app_name}/eval-sets/{eval_set_id}/add-session

5. The server responds with a 500 error and logs a stacktrace similar to:

   File "…/google/adk/cli/adk_web_server.py", line 952, in add_session_to_eval_set
       invocations = evals.convert_session_to_eval_invocations(session)
   File "…/google/adk/cli/utils/evals.py", line 52, in convert_session_to_eval_invocations
       return EvaluationGenerator.convert_events_to_eval_invocations(events)
   File "…/google/adk/evaluation/evaluation_generator.py", line 315, in convert_events_to_eval_invocations
       Invocation(
   File "…/pydantic/main.py", line 250, in __init__
       validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
   pydantic_core._pydantic_core.ValidationError: 1 validation error for Invocation
   user_content
     Input should be a valid dictionary or object to extract fields from
     [type=model_attributes_type, input_value='', input_type=str]
   

A more direct minimal example (conceptual) that shows the type mismatch:

from google.adk.evaluation.evaluation_generator import EvaluationGenerator
from google.adk.sessions.session import Session
from google.adk.events.event import Event
from google.genai.types import Content, Part

events = [
    Event(
        invocation_id="inv1",
        author="agent",  # no corresponding "user" event with this invocation_id
        content=Content(parts=[Part(text="agent-only event")]),
    )
]

session = Session(
    id="test-session",
    app_name="test-app",
    user_id="test-user",
    events=events,
)

# This leads to user_content="" for inv1, which fails validation.
EvaluationGenerator.convert_events_to_eval_invocations(events)
# or convert_session_to_eval_invocations(session)

Expected behavior

The eval endpoint should not crash with a Pydantic ValidationError when a session contains an invocation without a user-authored
event. Instead, I would expect one of the following:

• Invocations that have no author == "user" event are skipped or handled explicitly when converting to Invocation, or
• A valid google.genai.types.Content placeholder is constructed for such cases, or
• A clear, client-facing error is returned explaining that the session does not contain a valid user message for that invocation,
  rather than an internal 500 error.

Screenshots

Not applicable; this is a server-side error. The relevant stacktrace is provided above.

Desktop (please complete the following information):

• OS: macOS (Apple Silicon)

  Darwin5.1.0 Darwin Kernel Version 25.1.0: Mon Oct 20 19:32:41 PDT 2025; root:xnu-12377.41.6~2/
  RELEASE_ARM64_T6000 arm64

• Python version (python -V):

  Python 3.13.3

• ADK version (pip show google-adk):

  google-adk 1.19.0

Model Information:

• Are you using LiteLLM: No

• Which model is being used (e.g. gemini-2.5-pro):

  gemini-2.0-flash

Additional context

From reading the installed code, the behavior appears to come from EvaluationGenerator.convert_events_to_eval_invocations:

# google/adk/evaluation/evaluation_generator.py
for invocation_id, events in events_by_invocation_id.items():
    final_response = None
    user_content = ""          # default is an empty string
    invocation_timestamp = 0
    app_details = None
    ...
    for event in events:
        current_author = (event.author or _DEFAULT_AUTHOR).lower()
        if current_author == _USER_AUTHOR:  # _USER_AUTHOR = "user"
            user_content = event.content    # a Content object in the normal case
            invocation_timestamp = event.timestamp
            continue
    ...
    invocations.append(
        Invocation(
            invocation_id=invocation_id,
            user_content=user_content,  # may still be "" if no user event exists
            ...
        )
    )

Invocation.user_content is declared as genai_types.Content:

# google/adk/evaluation/eval_case.py
class Invocation(EvalBaseModel):
    invocation_id: str = ""
    user_content: genai_types.Content

If there is no author == "user" event for a given invocation_id, user_content remains a string (""), which is incompatible with
genai_types.Content and leads to the ValidationError.

A possible fix would be to:

• Initialize user_content as None instead of "", and
• Skip or specially handle invocations where no user event is found before constructing Invocation.

This would make the conversion from Session to eval invocations robust even when some invocations are internal/system-driven and do not have a direct user message.

[surajksharma07]

@lifeodyssey user_content is initialized as an empty string on line 280 of evaluation_generator.py but Invocation.user_content expects a Content type, causing a Pydantic validation error when no user events exist for an invocation.

The fix should initialize user_content = None instead and skip invocations without user content (since evaluations without user input aren't meaningful).

This is a legitimate bug affecting production web servers when sessions contain system-only invocations, and your suggested solution is spot-on.

Thanks!

[surajksharma07]

@wyf7107 Please have a look into it.