Supercharge Your LLM with the Fastest KV Cache Layer

libcapture is a multiplatform c++ library that allows to capture the display and the microphone audio.

A library for doing secure aggregation Hint-RLWE based

LLM KV cache compression made easy

Structured Outputs

A library for mechanistic interpretability of GPT-style language models

SΩI: Score-based O-INFORMATION Estimation

Karras et al. (2022) diffusion models for PyTorch

Official Repository of the paper "Let Them Drop: Scalable and Efficient Secure Federated Learning Solutions Agnostic to Client Stragglers "

llm.c, but in SYCL/Intel oneAPI!

Simple autodiff library built on NumPy, inspired by micrograd

Hackable and optimized Transformers building blocks, supporting a composable construction.

The official Meta Llama 3 GitHub site

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

Inference Llama 2 in one file of pure C

Official implementation of MINDE: Mutual Information Neural Diffusion Estimation

Hydra is a framework for elegantly configuring complex applications

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A high-throughput and memory-efficient inference and serving engine for LLMs

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

One-Line-of-Code Data Mollification Improves Optimization of Likelihood-based Generative Models (NeurIPS 2023)

BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

Extension of Dipy with Privacy Enhancing Technologies

Large Context Attention

A blazing fast inference solution for text embeddings models

Large Language Model Text Generation Inference

Python bindings for the Transformer models implemented in C/C++ using GGML library.