Skip to content

OpenAI接口兼容性不佳以及一些其他问题 #2722

Open
@icewool

Description

@icewool

1、OpenAI格式的接口无法在Cline中使用;包括VL和300B都无法正常工作;后台五明显报错日志;

2、后台日志过于简陋,无法观测性能指标;还是有什么特殊日志文件?
SGlang或VLLM都有显示token生成速度等功能,但是fastdeploy暂时没有找到;

3、并发10个情况下,很容易出现一些莫名其妙的回复:
如一直输出,查看后台有报错日志:

Traceback (most recent call last):
File "/usr/lib/python3.10/logging/init.py", line 1100, in emit
msg = self.format(record)
File "/usr/lib/python3.10/logging/init.py", line 943, in format
return fmt.format(record)
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/utils.py", line 62, in format
message = super().format(record)
File "/usr/lib/python3.10/logging/init.py", line 678, in format
record.message = record.getMessage()
File "/usr/lib/python3.10/logging/init.py", line 368, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/output/token_processor.py", line 153, in process_sampling_results
self._process_batch_output()
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/output/token_processor.py", line 282, in _process_batch_output
llm_logger.info(
Message: 'recovery stop signal found at task chatcmpl-4d7a5a03-02f4-4cb9-b63e-8d19e5394274-8a49a2d5-7dcc-44ea-96eb-113544957f4d'
Arguments: ('token_ids: [-3]',)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions