language_models: Add vision support for Copilot Chat models #30155
+80
−22
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cla-bot
bot
added
the
cla-signed
imumesh18
force-pushed
the
copilot-vision-support
branch
from
b784ef6 to
0f18fe5
Compare
maxdeviant
changed the title
lj3954
approved these changes
May 8, 2025
| text_content.push_str(text); | ||
| } | ||
| } | ||
| MessageContent::Image(image) => { |
There was a problem hiding this comment.
Suggested change
| MessageContent::Image(image) => { | |
| MessageContent::Image(image) if self.model.supports_vision() => { |
You can coalesce the if statement in the match expression, it makes it clearer that this case is not used otherwise.
JosephTLyons
pushed a commit
that referenced
this pull request
May 12, 2025
Problem Statement: Support for image analysis (vision) is currently restricted to Anthropic and Gemini models. This limits users who wish to leverage vision capabilities available in other models, such as Copilot, for tasks like attaching image context within the agent message editor. Proposed Change: This PR extends vision support to include Copilot models that are already equipped with vision capabilities. This integration will allow users within VS Code to attach and analyze images using supported Copilot models via the agent message editor. Scope Limitation: This PR does not implement controls within the message editor to ensure that image context (e.g., through copy-paste or attachment) is exclusively enabled or prompted only when a vision-supported model is active. Long term the message editor should have access to each models vision capability and stop the users from attaching images by either greying out the context saying it's not support or not work through both copy paste and file/directory search. Closes #30076 Release Notes: - Add vision support for Copilot Chat models --------- Co-authored-by: Bennet Bo Fenner <[email protected]>
JosephTLyons
pushed a commit
that referenced
this pull request
May 12, 2025
Problem Statement: Support for image analysis (vision) is currently restricted to Anthropic and Gemini models. This limits users who wish to leverage vision capabilities available in other models, such as Copilot, for tasks like attaching image context within the agent message editor. Proposed Change: This PR extends vision support to include Copilot models that are already equipped with vision capabilities. This integration will allow users within VS Code to attach and analyze images using supported Copilot models via the agent message editor. Scope Limitation: This PR does not implement controls within the message editor to ensure that image context (e.g., through copy-paste or attachment) is exclusively enabled or prompted only when a vision-supported model is active. Long term the message editor should have access to each models vision capability and stop the users from attaching images by either greying out the context saying it's not support or not work through both copy paste and file/directory search. Closes #30076 Release Notes: - Add vision support for Copilot Chat models --------- Co-authored-by: Bennet Bo Fenner <[email protected]>
dive
pushed a commit
to dive/zed
that referenced
this pull request
May 12, 2025
…stries#30155) Problem Statement: Support for image analysis (vision) is currently restricted to Anthropic and Gemini models. This limits users who wish to leverage vision capabilities available in other models, such as Copilot, for tasks like attaching image context within the agent message editor. Proposed Change: This PR extends vision support to include Copilot models that are already equipped with vision capabilities. This integration will allow users within VS Code to attach and analyze images using supported Copilot models via the agent message editor. Scope Limitation: This PR does not implement controls within the message editor to ensure that image context (e.g., through copy-paste or attachment) is exclusively enabled or prompted only when a vision-supported model is active. Long term the message editor should have access to each models vision capability and stop the users from attaching images by either greying out the context saying it's not support or not work through both copy paste and file/directory search. Closes zed-industries#30076 Release Notes: - Add vision support for Copilot Chat models --------- Co-authored-by: Bennet Bo Fenner <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem Statement:
Support for image analysis (vision) is currently restricted to Anthropic and Gemini models. This limits users who wish to leverage vision capabilities available in other models, such as Copilot, for tasks like attaching image context within the agent message editor.
Proposed Change:
This PR extends vision support to include Copilot models that are already equipped with vision capabilities. This integration will allow users within VS Code to attach and analyze images using supported Copilot models via the agent message editor.
Scope Limitation:
This PR does not implement controls within the message editor to ensure that image context (e.g., through copy-paste or attachment) is exclusively enabled or prompted only when a vision-supported model is active. Long term the message editor should have access to each models vision capability and stop the users from attaching images by either greying out the context saying it's not support or not work through both copy paste and file/directory search.
Closes #30076
Release Notes: