Essay · Kunj Shah

Building GPT-2 from First Principles: Tokenization to Autoregressive Generation

A week-by-week journey implementing GPT-2 from scratch — from byte-pair tokenization through multi-head attention to full autoregressive text generation. No frameworks, just NumPy and PyTorch.

All projects · Writing · About · Contact