1. Python environment

This book targets Python 3.11 or 3.12. Earlier versions will hit type-syntax problems; newer versions may have torch wheel availability issues.

The short version

# Check what you have
python --version

# Create an isolated env (any of these works; pick one)
python -m venv .venv && source .venv/bin/activate     # macOS / Linux
python -m venv .venv && .venv\Scripts\activate        # Windows PowerShell

# OR with uv (fast, recommended if you don't already have a venv tool)
uv venv && source .venv/bin/activate

Then proceed to 02_installing-dependencies.md.

If you don't have Python at all

• macOS: brew install python@3.12
• Linux (Debian/Ubuntu): sudo apt install python3.12 python3.12-venv
• Windows: Download the installer from https://www.python.org/downloads/ — check "Add Python to PATH" during install.

If you're using conda / mamba

conda create -n uth python=3.12
conda activate uth

That's it for environments. You don't need notebook tooling, jupyter, or anything else yet — this repo runs .py files directly.

2. Installing dependencies

Core install (every project)

pip install -r requirements.txt

This gives you torch, numpy, matplotlib, tiktoken, pytest — enough to run roughly Projects 1–7 and to run every project's smoke tests.

Dev install (if you'll modify code or run linters)

pip install -e ".[dev]"

Adds ruff, pyright, pytest-xdist, jupytext.

Full install (heavier projects: datasets, transformers, FAISS, mixed-precision)

pip install -e ".[full]"

Adds datasets, transformers, huggingface_hub, faiss-cpu, mmh3, pandas, tqdm. Some later projects (8, 10, 14, 15, 28) need these.

Per-project extras

A few projects need exotic dependencies (flash-attn, bitsandbytes, llama-cpp-python). Those live in projects/NN_slug/requirements-extra.txt and you install them only if you do that project:

pip install -r projects/16_quantization-and-deployment/requirements-extra.txt

CUDA / CPU torch install

The default requirements.txt line torch>=2.2 gets you whatever torch wheel pip picks for your platform — usually the CUDA build on Linux/Windows with a GPU, and the CPU build on macOS.

If you want to force the CPU build (smaller, faster install, no GPU needed):

pip install --index-url https://download.pytorch.org/whl/cpu torch

If you want a specific CUDA version (e.g. CUDA 12.1):

pip install --index-url https://download.pytorch.org/whl/cu121 torch

The full matrix is at https://pytorch.org/get-started/locally/.

Verifying the install

python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
pytest --collect-only

If the second command reports collected tests without errors, you're done.

3. GPU and hardware tiers

This is the most-asked question this book gets. The honest answer: you can do most of the projects on a laptop — the rest have proxy versions designed for limited hardware.

Three tiers

Tier	Hardware	What you can run	Projects
Laptop / CPU	Any laptop, no GPU required	Smoke + unit tests; tiny configs (--tiny flag); full code for Projects 1–7	1–7, plus tier-1 tests for every other project
One consumer GPU	8–24 GB VRAM (RTX 3070 / 4070 / 4090, M-series Mac with mps)	Most full labs at modest scale; pretraining is short; quantization and serving are realistic	8–16, 18, 22–25, 28–30
Cloud rental / data center	Single A100 / H100, or short rental on Lambda / RunPod / Modal	The full versions of: scaled pretraining, RLHF, large MoE, long-context fine-tuning	8, 10, 11, 14, 15, 17, 21, 27, 31

What --tiny means

Every project's build.py accepts a --tiny flag. With it, the project runs on a proxy configuration:

• n_layer=2, n_head=2, d_model=32
• Synthetic 1 MB dataset substituted for FineWeb / HF datasets
• 50 steps instead of 10,000
• Should complete on CPU in under 60 seconds

This is what the CI tests run against. It's also what readers without a GPU should run for the full project — you learn the structure without the wait.

python projects/05_your-gpt-from-a-blank-file/build.py --tiny

For the full version (only if you have the hardware):

python projects/05_your-gpt-from-a-blank-file/build.py --full

"But I want to do the real thing"

If you don't have a GPU and you really want to run the full version of a project, three options:

1. Cloud rental — by the hour. Lambda, RunPod, Vast.ai, Modal, Paperspace. An A100 runs about $1–2/hour. Most full labs in this book finish in under 3 hours.
2. Colab Pro (low-end GPUs). A T4 or L4 will handle Projects 8, 14, 16, 22 at modest scale. Slower than an A100 but ~10× cheaper.
3. HuggingFace Spaces / Inference Endpoints. For projects 14–16 where you're fine-tuning rather than pretraining, a hosted endpoint can be cheaper than renting GPUs by the hour.

What you should NOT do

• Don't buy a GPU specifically for this book. The --tiny proxy versions are designed to give you the same structural insight without the hardware. Rent if you need scale.
• Don't try to run the full FineWeb pretraining on a laptop. It will not finish. --tiny substitutes a 1 MB synthetic corpus that's enough to show the pipeline working.
• Don't run Project 16 (quantization to GGUF) without checking disk space. A full INT4 export of a 1.5B parameter model is ~800 MB. Multiple variants × multiple projects = your models/ folder fills up fast.

4. Running .py files (and converting to notebooks if you prefer)

This repo ships .py files, not .ipynb notebooks. The reasons:

• .py runs cleanly in CI; notebooks don't (output cells churn the diff).
• .py works in any editor; notebooks lock you into Jupyter / VSCode notebook UI.
• .py plays well with version control; notebook JSON does not.

But you can still get a notebook experience if that's how you prefer to learn.

Running a project's main script

# From the repo root:
python projects/01_the-learning-machine/build.py

Common flags every project supports:

Flag	What it does
(none)	Default config — runs on whatever hardware you have
--tiny	Proxy config: tiny model, synthetic data, ~60s on CPU
--full	Full lab — requires the hardware described in 03_gpu-and-hardware-tiers.md
--seed N	Override the default seed (0) for reproducibility experiments
--output-dir PATH	Where to write outputs (defaults to projects/NN_slug/outputs/)

Running the BREAK IT experiment

Every project has a break_it.py next to build.py. The BREAK IT runs the same code with one mechanism deliberately disabled or sabotaged — you compare the output to build.py to learn why that mechanism is there:

python projects/01_the-learning-machine/break_it.py

Running tests

# All projects
pytest

# One project
pytest projects/01_the-learning-machine/tests/

# Only fast tests (skip @pytest.mark.slow and @pytest.mark.gpu)
pytest -m "not slow and not gpu"

# Including slow tests
UTH_RUN_SLOW=1 pytest

Converting .py to .ipynb

If you prefer to step through cells in Jupyter, use jupytext. Install once:

pip install jupytext

Then convert any project's build.py:

jupytext --to ipynb projects/01_the-learning-machine/build.py
# produces projects/01_the-learning-machine/build.ipynb
jupyter lab projects/01_the-learning-machine/build.ipynb

The conversion treats #%% lines as cell breaks. If you want cleaner cell boundaries, edit the .py to add #%% markers where you'd like a new cell — jupytext respects them on re-conversion.

To keep .py and .ipynb synced as you edit the notebook:

jupytext --set-formats py:percent,ipynb projects/01_the-learning-machine/build.py

After that, saving the .ipynb auto-updates the .py and vice versa.

Why we don't ship notebooks directly

We considered shipping both formats. Three reasons we didn't:

1. Output cell churn. A notebook re-run changes random cell outputs and version-controlled metadata. The diffs become noise.
2. Pedagogical drift. Notebooks reward you for executing top-to-bottom. The book's BREAK IT philosophy needs you to comment lines out, re-run subsections, and compare. That's friction in a notebook UI.
3. Honest deps. A .py file declares its imports at the top. A notebook hides them across cells. The book is about understanding what's actually happening — that starts with imports.

If you disagree, the jupytext recipe above gets you a notebook in 10 seconds. We just don't make it the default.

5. Datasets and checkpoints

Most projects in this book fetch external data. None of it is bundled with the repo — the files are too large and would balloon the clone size. This page explains where each dataset comes from, how to fetch it, and how the --tiny flag lets you skip the fetch entirely while you're learning.

The three tiers of data access

Tier	What you get	Project supports
Tiny (default)	1 MB synthetic substitute; runs on CPU in <60s	Every project, via --tiny flag
Sample	A few hundred MB; representative slice of the real dataset	Most projects that train (8, 10, 14, 24)
Full	The actual dataset (often GBs)	Set env var UTH_FULL_DATA=1 to enable

The --tiny mode is the default for a reason: it lets you learn the structure of the project without committing to a multi-GB download. Switch up only when you're ready to run the full lab.

Datasets used in this book

FineWeb-EDU (Project 8: Pretraining)

• Source: https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu
• Full size: ~5 TB (the book uses a 1B-token sample slice; ~5 GB)
• Sample slice: ~500 MB
• Tiny substitute: synthetic web-style text generated at runtime

# Sample slice (default for --full)
pip install datasets
huggingface-cli login   # if first time

# Tiny (default — no download)
python projects/08_pretraining-on-the-real-web/build.py

TinyStories (Projects 1–5)

• Source: https://huggingface.co/datasets/roneneldan/TinyStories
• Full size: ~1 GB
• Why we use it: Small enough to actually fit in your laptop's RAM, varied enough that next-character prediction produces interesting results.

WikiText-103 (Projects 6, 7, 13)

• Source: https://huggingface.co/datasets/wikitext
• Full size: ~500 MB

GSM8K (Project 15: Reward Models + RLHF)

• Source: https://huggingface.co/datasets/gsm8k
• Full size: ~10 MB
• Small enough that the --tiny and --full runs use the same data.

Custom preference datasets (Projects 14, 15)

Generated locally by the project's own scripts. No external download.

Checkpoints used in this book

nanochat reference checkpoint (Projects 6, 8, 12)

• Source: referenced in chapter prose; the book uses Karpathy's nanochat as a comparison point.
• Where to get it: git clone https://github.com/karpathy/nanochat — the model weights are not in this repo.
• The relevant projects gracefully fall back to a re-implementation if nanochat is absent.

llama.cpp GGUF models (Project 16: Quantization)

• Source: https://huggingface.co/models?library=gguf
• Full size: 500 MB – 4 GB per model depending on quantization level.
• The project lets you point at any GGUF file via --model PATH; doesn't ship any.

Embedding model checkpoints (Projects 28, 29: Retrieval)

• Source: https://huggingface.co/sentence-transformers
• The default sentence-transformers/all-MiniLM-L6-v2 is ~80 MB and downloads on first use.

Cache locations

By default, HuggingFace caches go to:

• Linux / macOS: ~/.cache/huggingface/
• Windows: C:\Users\<you>\.cache\huggingface\

To point them elsewhere (e.g., to an external drive):

export HF_HOME=/mnt/external/huggingface
# Windows PowerShell:
$env:HF_HOME = "D:\huggingface"

The repo's .gitignore excludes data/, checkpoints/, models/, .cache/, *.gguf, *.bin, *.safetensors so any locally-fetched files stay out of git.

What happens if a download fails

Every project that fetches data wraps the fetch with a clear error message — e.g. "Could not reach HuggingFace; rerun with --tiny to use the synthetic substitute." The synthetic substitute is intentionally limited (it won't produce a quality model) but it's enough to verify the pipeline runs and the shapes are right.