Skip to content

Conversation

dmchoiboi
Copy link
Collaborator

Pull Request Summary

After deploying #639 to all endpoints, we can remove the model.name override we do here

What is this PR changing? Why is this change being made? Any caveats you'd like to highlight? Link any relevant documents, links, or screenshots here if applicable.

Test Plan and Usage Guide

How did you validate that your PR works correctly? How do you run or demo the code? Provide enough detail so a reviewer can reasonably reproduce the testing procedure. Paste example command line invocations if applicable.

Copy link
Contributor

@seanshi-scale seanshi-scale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would we need to redeploy everything on llm engine? (including people's private models and all of that)

@dmchoiboi
Copy link
Collaborator Author

dmchoiboi commented Oct 16, 2024

would we need to redeploy everything on llm engine? (including people's private models and all of that)

This would only apply for endpoints that support oia compatible APIs, which is the plan

@dmchoiboi dmchoiboi merged commit 7cb43cd into main Oct 16, 2024
7 checks passed
@dmchoiboi dmchoiboi deleted the dmchoi/remove-model-name-overridea branch October 16, 2024 19:44
dmchoiboi added a commit that referenced this pull request Oct 17, 2024
dmchoiboi added a commit that referenced this pull request Oct 17, 2024
dmchoiboi added a commit that referenced this pull request Oct 17, 2024
dmchoiboi added a commit that referenced this pull request Oct 17, 2024
* Revert "Remove model name override (#641)"

This reverts commit 7cb43cd.

* Remove 'Annotated' type usage - python 3.8/pydantic doesn't like it

* Add request id on batch completions error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants