ChatGPT calling MCP does not receive the errors

[lucfranken]

Describe the bug

When using the ChatGPT Response API to call a tool:

It does not understand or see the errors returned.

The SDK does respond correctly to errors, like example of missing parameters:

"MCP error -32602: MCP error -32602: Invalid arguments for tool get-hour-totals: [
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "startDate"
    ],
    "message": "Required"
  },
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "endDate"
    ],
    "message": "Required"
  }
]"

To Reproduce

1. Call a tool from the MCP inspector -> should show an error
2. Let ChatGPT call the tool:

Expected behavior

Show the errors.

Logs

Tested with Playground:

Tested with MCP Inspector:

Additional context

I have the idea this behavior has changed since a short time ago, cannot specify exactly how long.

Not sure if it might be more strict now on combining text and json in a single message?

Important detail maybe: We for example run into this if ChatGPT generates a fictive parameter, we have like an optional userId field. ChatGPT just filled in: user-123 to fill the box. Off course that goes wrong but it is very hard to find as you don't see the MCP call.