Skip to content

Conversation

@pazevedo-hyland
Copy link
Contributor

@pazevedo-hyland pazevedo-hyland commented Jun 3, 2025

This PR adds support for DocumentBlock content in the LiteLLM integration, enabling users to send PDF and other document files to LLM providers through LiteLLM.

Changes Made

llm = LiteLLM(
    "litellm_proxy/bedrock-claude-3-7-sonnet-20250219-v1:0",
    api_base="...",
    api_key="sk-...",
)

document_block = DocumentBlock(
    data=pdf_data_in_bytes,
    document_mimetype="application/pdf",
    title="test_document.pdf",
)
text_block = TextBlock(text="Please analyze this document.")

message = ChatMessage(role=MessageRole.USER, content=[document_block, text_block])

# Send the message
chat_response = llm.chat([message])

New Features

  • Added DocumentBlock support: Documents are now converted to LiteLLM expected format using type file with base64-encoded file_data.
  • Enhanced test coverage: Added comprehensive tests for DocumentBlock conversion and integration.

Testing

  • All existing tests pass
  • New DocumentBlock tests added and passing
  • Verified proper base64 encoding/decoding
  • Confirmed LiteLLM format compatibility
  • Tested with claude 3.7 sonnet via litellm proxy

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jun 3, 2025
@pazevedo-hyland
Copy link
Contributor Author

@AstraBert take a look at this if possible

@AstraBert
Copy link
Member

@pazevedo-hyland yep, taking a look at this tomorrow, thanks for the integration🙏

@AstraBert AstraBert self-assigned this Jun 3, 2025
@AstraBert AstraBert self-requested a review June 3, 2025 20:56
@pazevedo-hyland
Copy link
Contributor Author

@AstraBert you have any clue why the imports are failing on unit tests?

@AstraBert
Copy link
Member

Hey @pazevedo-hyland, it's a uv problem: the version of llama-index-core listed in uv.lock is 0.12.33.post1, which didn't have DocumentBlock in it. I'll take care of changing it, dw :)

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jun 4, 2025
@AstraBert AstraBert enabled auto-merge (squash) June 4, 2025 10:52
@AstraBert AstraBert merged commit 41a7158 into run-llama:main Jun 4, 2025
10 checks passed
@colca colca mentioned this pull request Jun 9, 2025
18 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants