Popular repositories Loading

1. xInfer-ai xInfer-ai Public

   Config files for my GitHub profile.

2. ktransformers ktransformers Public

   Forked from kvcache-ai/ktransformers

   A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

   Python

3. fastllm fastllm Public

   Forked from ztxz16/fastllm

   纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

   C++

4. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a fast serving framework for large language models and vision language models.

   Python

5. PQCache PQCache Public

   Forked from HugoZHL/PQCache

   Python

6. llumnix llumnix Public

   Forked from AlibabaPAI/llumnix

   Efficient and easy multi-instance LLM serving

   Python