feat: add max_retries parameter to structured_output for self-healing

[kaleko]

• Add max_retries parameter to structured_output() and structured_output_async()
• Implement retry loop that catches ValidationError and ValueError
• Add telemetry tracking for retry attempts
• Add tests for retry functionality
• Maintain backwards compatibility with max_retries=0 as default

Fixes #1014

Description

This PR adds a max_retries parameter to the structured_output() and structured_output_async() methods to enable automatic self-healing retry attempts when validation errors occur.

When max_retries > 0, the agent will automatically retry structured output generation if it encounters a ValidationError or ValueError. On each retry, the error message is appended to the temporary messages (not conversation history) so the model can self-correct based on what went wrong.

Key changes:

• Added max_retries: int = 0 parameter (default maintains backwards compatibility)
• Implemented retry loop that catches ValidationError and ValueError
• On retry, sends error context to the model: "Try again to generate a structured output. Your previous attempt failed with this exception: {e}"
• Added telemetry tracking via span attributes and events for retry attempts
• Added comprehensive tests for retry functionality

Related Issues

Fixes #1014

Documentation PR

No documentation PR needed at this time. The feature is self-explanatory through docstrings and follows existing patterns.

Type of Change

New feature

Testing

All tests pass successfully:

• Added 3 new tests specifically for retry functionality:

  • test_agent_structured_output_with_retry_on_validation_error() - validates retry on ValidationError
  • test_agent_structured_output_with_retry_on_value_error() - validates retry on ValueError
  • test_agent_structured_output_retry_exhausted() - validates exception raised after exhausting retries

• Updated existing test to expect new telemetry attribute

• All 1136 tests pass (including 39 structured_output tests across the codebase)

• No warnings introduced

• I ran hatch run prepare

Checklist

• I have read the CONTRIBUTING document
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[kaleko]

@zastrowm could you approve the workflow on this PR?