Skip to content

500 InternalServerError from Google Generative AI client with no retry/backoff #1014

Open
@EngineerAbdullahIqbal

Description

@EngineerAbdullahIqbal

When calling the Generative AI endpoint via openai-agents-sdk (via AsyncOpenAI + OpenAIChatCompletionsModel), we occasionally receive a raw 500 with no automatic retry or helpful diagnostics:

openai.InternalServerError: Error code: 500 - [{'error': {
  'code': 500,
  'message': 'An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting',
  'status': 'INTERNAL'
}}]

This bubbles up directly to user code, with no built‑in exponential‑backoff or payload dump.


Steps to Reproduce:

  1. Install latest openai and openai-agents-sdk.
  2. Initialize SDK with Gemini-Flash endpoint.
  3. Send any valid prompt via Runner.run(...) repeatedly in a loop.
  4. Observe occasional 500 errors with no retry.

Expected Behavior:

  • SDK should implement a configurable retry/backoff policy on HTTP 500s (e.g. up to N attempts, doubling delay).
  • On fatal server errors, expose request payload (or at least message lengths and roles) to help diagnose malformed inputs.

Actual Behavior:

  • A single raw InternalServerError exception is raised.
  • No retry, no backoff, no context about which request triggered it.

Suggested Fixes:

  1. Wrap calls to openai_client.chat.completions.create in a common retry decorator (e.g. using tenacity or custom backoff).
  2. On failure, log the HTTP request body (without API key) at debug level.
  3. Expose retry/backoff parameters via RunConfig or client constructor.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions