regression训练的模型如何部署

[uRENu]

参考如下示例部署失败，报错requests.exceptions.HTTPError: The model does not support Chat Completions API
https://github.com/modelscope/ms-swift/blob/07bd7b463f1588362388c389dc6bdb4570bea04e/examples/train/seq_cls/regression/deploy.sh

[Jintao-Huang]

请拉一下main分支，已经修复这个问题了

[uRENu]

请拉一下main分支，已经修复这个问题了
拉了最新的swift main分支还是有同样问题requests.exceptions.HTTPError: The model does not support Chat Completions API，以下是调用代码

from swift.llm import InferClient, InferRequest
openai_api_key = ""
openai_api_base = "http://XXXXX:8080/v1"
engine = InferClient(
api_key=openai_api_key,
base_url=openai_api_base,
)

models = engine.models
print(f'models={models}')
messages = [{
'role': 'user',
'content': content}]
resp_list = engine.infer([InferRequest(messages=messages)])
print(f'response: {resp_list}')

[Jintao-Huang]

我没有理解，是有什么报错吗

[uRENu]

我没有理解，是有什么报错吗

engine.infer报错requests.exceptions.HTTPError: The model does not support Chat Completions API

[Jintao-Huang]

报错截图看看

[uRENu]

报错截图看看

[uRENu]

报错截图看看

在VLLM里面看了下，部署和调用方式如下：
1、部署
python -m vllm.entrypoints.openai.api_server --model /data/mymodel --port 8080
2、调用

import requests

def post_http_request(prompt: dict, api_url: str) -> requests.Response:
headers = {"User-Agent": "Test Client"}
response = requests.post(api_url, headers=headers, json=prompt)
return response

def infer_instance(content):
api_url = "http://myhost:8080/pooling"
# Input like Completions API
prompt = {"model": model, "input": content}
# print(prompt)
pooling_response = post_http_request(prompt=prompt, api_url=api_url)
print(pooling_response.json()['data'][0]['data'])