Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 500 universities from 70 countries including Stanford, MIT, Harvard, and Cambridge.

NanoGPT (124M) in 3 minutes

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

Code for BLT research paper

Code for 'LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders'

Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI

A pure NumPy implementation of Mamba.

Normalized Transformer (nGPT)

Video+code lecture on building nanoGPT from scratch