A high-throughput and memory-efficient inference and serving engine for LLMs

Port of OpenAI's Whisper model in C/C++

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discr…

DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported …

Local LLM Server with NPU Acceleration

A pipeline for LLM knowledge distillation

Native port of Redis for Windows,it can be installed as service.

Faster Whisper transcription with CTranslate2

A graphical tool that can extract and replace files from encrypted and non-encrypted iOS backups

Chocolatey in Windows Server Core Docker image

A series of large language models trained from scratch by developers @01-ai

Shader Connect is a lightweight C++17 command-line tool for converting GLSL/HLSL shader code to optimized GLSL/ESSL/HLSL/MetalSL/SPIR-V.

🔊 Text-Prompted Generative Audio Model

High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model

Python bindings for whisper.cpp

Multiple NVIDIA GPUs or Apple Silicon for Large Language Model Inference?

Synthetic Patient Population Simulator

gpt4all: run open-source LLMs anywhere

Postgres extension and service for automated failover and high-availability

A guide on how to host whisper on AWS free tier

Java DSL for easy testing of REST services

Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, OPT, and GALACTICA.

The release repo for "Vicuna: An Open Chatbot Impressing GPT-4"

Fork from https://github.com/hpcaitech/ColossalAI/tree/main/applications/Chat

Containers for machine learning

Locally run an Instruction-Tuned Chat-Style LLM

The simplest way to run LLaMA on your local machine