Skip to content

Conversation

wbrennan899
Copy link

@wbrennan899 wbrennan899 commented Jun 25, 2025

Description

This PR adds TwelveLabs model provider integration to the Strands SDK, enabling both semantic video search and video understanding capabilities. The implementation provides two specialized model providers:

  • TwelveLabsSearchModel: Semantic video search using the Marengo model
  • TwelveLabsPegasusModel: Video understanding and Q&A using the Pegasus model

Key Features

TwelveLabsSearchModel:

  • Search TwelveLabs video indexes using natural language queries
  • Search video and audio content with configurable modalities (visual, audio)
  • Support for both clip-level and video-grouped results

TwelveLabsPegasusModel:

  • Upload and index videos with automatic deduplication
  • Chat with existing videos in your TwelveLabs index for Q&A

Related Issues

Documentation PR

Type of Change

New feature

Testing

How have you tested the change?

  • I ran hatch run prepare - All linting and formatting checks pass
  • Unit tests: 26/26 tests pass for the TwelveLabs model implementation
  • Integration with existing SDK patterns verified
  • Error handling tested for all TwelveLabs API error codes
  • Type checking with mypy passes
  • Code formatting with ruff passes

The integration tests failed due to missing API credentials for AWS/OpenAI services, which is expected and unrelated to the TwelveLabs implementation.

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@zastrowm zastrowm added the area-provider Related to model providers label Jul 22, 2025
@james-le-twelve-labs
Copy link

@zastrowm @awsarron Will this PR be merged soon to Strands' Python SDK repo?

@awsarron awsarron assigned awsarron and unassigned awsarron Aug 15, 2025
@Unshure Unshure assigned yonib05 and unassigned awsarron Sep 18, 2025
@yonib05
Copy link
Member

yonib05 commented Sep 19, 2025

Closing this PR in favor of this tools PR strands-agents/tools#120 given the Twelve Labs Models fit better as tools.

@yonib05 yonib05 closed this Sep 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-provider Related to model providers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants