Under the Hood

Build it. Break it. Measure it. — not a tagline. It is the entire epistemology of the book.

This is the runnable code companion for the book Under the Hood. Each of the36 projects has its own folder, with a runnable build.py, a break_it.py experiment, per-step reference files, and captured outputs.

You can do the entire book by reading the book and typing the code yourself — that's the recommended path. But when you get stuck, or want to compare your code to a reference, this site is where you look.

Who this is for

• Engineers who can ship software but feel like LLM internals are wizardry happening behind an API.
• Researchers who want a working mental model of the full stack before reading their hundredth attention paper.
• Students who learned PyTorch from notebooks and want to know what a training run actually does.
• Practitioners tuning, deploying, or fine-tuning models who keep hitting failure modes they cannot explain.

Philosophy

• Build it because reading is not understanding. Understanding is being able to reproduce.
• Break it because a working system tells you what it does. A broken system tells you why it has to be that way.
• Measure it because intuition without numbers is the most expensive failure mode in machine learning.

Modern LLMs are not magic. They are a stack of carefully-tuned engineering decisions, each one earned by a specific failure that someone, somewhere, had to debug at 2am. This book makes you the person who debugged it.

The code vs. the book

The code in this repository is released under the MIT License. The book text (PDF / EPUB, distributed via Leanpub) is copyrighted separately.

Citation

@book{kumaresan2026underthehood,
  author    = {Kumaresan, Ramchand},
  title     = {Under the Hood: Build Every Layer of a Large Language Model from Scratch},
  year      = {2026},
  publisher = {Leanpub},
  url       = {https://leanpub.com/under-the-hood}
}