[COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

[ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding

Repository of the paper "Accelerating Transformer Inference for Translation via Parallel Decoding"

Fast inference from large lauguage models via speculative decoding

Codes for our paper "Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation" (EMNLP 2023 Findings)

📰 Must-read papers and blogs on Speculative Decoding ⚡️