Skip to content

Database workshop · 7 chapters · 88 minutes

Pull the plug.

Build a database from scratch, then crash it at every byte it writes. An append-only log, an index, checksums, fsync and compaction: the ideas inside every real database, in under 200 lines of Python you can read in one sitting.

  • Append-only log
  • Hash index
  • Checksums
  • Torn writes
  • fsync
  • Group commit
  • Compaction
Start the workshopView the code
stash plugcrash at every byte of 100 writes
naive JSON file0.2%
kept your data at 144 of 66,400 crash points
Stash100.0%
kept your data at 4,754 of 4,754 crash points
Then kill a real writer process 5 times: 376,017 writes acked, 0 lost.

What you run and read

The loglab/stash/record.pyAppend, never overwrite. Checksums on every record.
The indexlab/stash/store.pyEvery key to one seek. Rebuilt on open.
The pluglab/stash/plug.pyCrash at every byte, or kill a real writer.
The benchlab/stash/bench.pyWhat flush, fsync and batching cost.

7 short chapters

Each one is a few minutes of reading, one thing to run or break, and an exercise. By the end, words like WAL, fsync and compaction mean something you have built.

  1. 01The one-line database10 min
  2. 02Append, don't overwrite12 min
  3. 03An index in memory12 min
  4. 04Pull the plug15 min
  5. 05What "saved" means15 min
  6. 06Compaction12 min
  7. 07The trade-offs, and real databases12 min

Run it in a minute

You need uv and git. The first command crashes two databases at every byte and counts what survives. No other installs needed.

git clone https://github.com/jars-demo/build-a-database-demo.git
cd build-a-database-demo/lab
uv sync
uv run python -m stash plug