Essay · Kunj Shah
Building GPT-2 from First Principles: Tokenization to Autoregressive Generation
A week-by-week journey implementing GPT-2 from scratch — from byte-pair tokenization through multi-head attention to full autoregressive text generation. No frameworks, just NumPy and PyTorch.
All projects · Writing · About · Contact