Skip to content

fix llama-cpp-python[server] issues #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 27, 2025
Merged

Conversation

ljm625
Copy link

@ljm625 ljm625 commented Jan 27, 2025

After testing on llama-cpp-server[python]
I noticed such issue below:

Exception: Llama.tokenize() missing 1 required positional argument: 'text'
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/llama_cpp/server/errors.py", line 173, in custom_route_handler
    response = await original_route_handler(request)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    return await dependant.call(**values)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/llama_cpp/server/app.py", line 526, in create_chat_completion
    return await run_in_threadpool(llama.create_chat_completion, **kwargs)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
    return await future
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 943, in run
    result = context.run(func, *args)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/llama_cpp/llama.py", line 2011, in create_chat_completion
    return handler(
  File "/opt/homebrew/Caskroom/miniconda/base/envs/llama/lib/python3.10/site-packages/llama_cpp/llama_chat_format.py", line 596, in chat_completion_handler
    prompt = llama.tokenize(
TypeError: Llama.tokenize() missing 1 required positional argument: 'text'

After the PR the issue is gone and the llama-cpp-python[server] working as expected

@JamePeng JamePeng merged commit e9edb78 into JamePeng:main Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants