Skip to content

feat(bedrock): async client support via asyncio.to_thread #1615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

duf59
Copy link

@duf59 duf59 commented Jun 18, 2025

Description

This PR improves the async support for AWS Bedrock clients in Instructor.

  • Uses asyncio.to_thread to wrap the synchronous boto3 Bedrock client, enabling non-blocking async usage.
  • Updates the docstring for clarity.
  • Adds documentation and a self-contained async example in the Bedrock integration guide.

Changes

  • Refactored async_wrapper to use asyncio.to_thread
  • Improved docstring for from_bedrock
  • Added async usage documentation in docs/integrations/bedrock.md

Testing

  • Manual review of async and sync usage
  • Linting and formatting pass

This PR was written by Cursor


Important

Adds async support for AWS Bedrock clients using asyncio.to_thread and updates documentation with async examples.

  • Async Support:
    • Uses asyncio.to_thread in from_bedrock() in client_bedrock.py to enable async usage of AWS Bedrock clients.
    • Adds async_wrapper function to handle async operations.
  • Documentation:
    • Adds async usage example in docs/integrations/bedrock.md.
    • Updates docstring for from_bedrock() to clarify async behavior.
  • Testing:
    • Manual review of async and sync usage.
    • Linting and formatting pass.

This description was created by Ellipsis for 3aa866e. You can customize this summary. It will automatically update as commits are pushed.

duf59 added 2 commits June 18, 2025 20:25
AsyncInstructor now uses asyncio.to_thread to run the synchronous boto3 Bedrock client in a thread, enabling non-blocking async usage. Updated docstring for clarity.
…tegration

Documented how Instructor enables async support for boto3 Bedrock client using asyncio.to_thread, with a self-contained example and clear explanation.
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Looks good to me! 👍

Reviewed everything up to 3aa866e in 59 seconds. Click for details.
  • Reviewed 89 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. docs/integrations/bedrock.md:56
  • Draft comment:
    Great async example! Consider adding a brief note about potential thread-pool overhead or thread-safety limitations when wrapping blocking calls with asyncio.to_thread.
  • Reason this comment was not posted:
    Confidence changes required: 33% <= threshold 85% None
2. instructor/client_bedrock.py:85
  • Draft comment:
    Using asyncio.to_thread to wrap client.converse is clear. Consider adding error handling (e.g., try/except) to capture exceptions from the threaded call for better debugging.
  • Reason this comment was not posted:
    Confidence changes required: 33% <= threshold 85% None

Workflow ID: wflow_pjHVgpLz8SHBPBAE

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant