Added support for the "think" for Ollama #3386
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1. Added the `think` field to Ollama's `ChatRequest`
2. Added the `thinking` field to Ollama's `Message`
3. Added the `think` property to `OllamaOptions`, allowing users to specify whether to enable or disable thinking
Signed-off-by: Sun Yuhan <[email protected]>
Signed-off-by: Sun Yuhan <[email protected]>
…fault behavior, thereby ensuring compatibility with older versions of Ollama calls. Signed-off-by: Sun Yuhan <[email protected]>
|
@tzolov @ilayaperumalg @markpollack Could you please help review this PR? Thank you. |
…tainer image version of ollama to 0.9.0 Signed-off-by: Sun Yuhan <[email protected]>
|
Yes, we will review. Thanks |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As mentioned in issue: #3383 , Ollama added support for "think" in its latest 0.9.0 version:
https://github.com/ollama/ollama/releases
https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-chat-completion.
This PR implements support for that attribute and includes the following key changes:
thinkfield to Ollama'sChatRequestthinkingfield to Ollama'sMessagethinkproperty toOllamaOptions, allowing users to specify whether to enable or disable thinkingActually, there is currently another issue:
As stated in Ollama's API documentation here, during requests to Ollama, the
messagefield supports sending the model's own reasoning (thoughts) back to it. However,AssistantMessagedoes not currently support transmitting this field, which means the model will not be aware of its previous thoughts.Therefore, perhaps we need to add a specialized
Messageimplementation for Ollama, such asOllamaAssistantMessage. I'm not sure whether this would be considered a significant change.