Description
1、OpenAI格式的接口无法在Cline中使用；包括VL和300B都无法正常工作；后台五明显报错日志；
2、后台日志过于简陋，无法观测性能指标；还是有什么特殊日志文件？
SGlang或VLLM都有显示token生成速度等功能，但是fastdeploy暂时没有找到；
3、并发10个情况下，很容易出现一些莫名其妙的回复：
如一直输出，查看后台有报错日志：
Traceback (most recent call last):
File "/usr/lib/python3.10/logging/init.py", line 1100, in emit
msg = self.format(record)
File "/usr/lib/python3.10/logging/init.py", line 943, in format
return fmt.format(record)
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/utils.py", line 62, in format
message = super().format(record)
File "/usr/lib/python3.10/logging/init.py", line 678, in format
record.message = record.getMessage()
File "/usr/lib/python3.10/logging/init.py", line 368, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/output/token_processor.py", line 153, in process_sampling_results
self._process_batch_output()
File "/usr/local/lib/python3.10/dist-packages/fastdeploy/output/token_processor.py", line 282, in _process_batch_output
llm_logger.info(
Message: 'recovery stop signal found at task chatcmpl-4d7a5a03-02f4-4cb9-b63e-8d19e5394274-8a49a2d5-7dcc-44ea-96eb-113544957f4d'
Arguments: ('token_ids: [-3]',)