How databases
survive a power cut.

Writing data in place is a gamble you eventually lose.

Here is the naive way to store a key-value pair: keep a file, find the row, overwrite it. It works perfectly right up until the instant it doesn't.

Say user:1 currently has a balance of 500 and we want to set it to 400. On disk that's three bytes to overwrite. Watch what a crash in the middle of those three bytes does:

There is no safe moment. Overwriting is destructive: the old value is gone before the new one has fully arrived, so an interrupted write leaves a value that was never correct at any point in time. Databases call this a torn write.

Write down what you're about to do, before you do it.

That's the whole idea. Keep a second file — the log — that only ever grows at the end. Before touching the real data, append a record describing the change and force it to disk. Only then apply it.

Appending is the key word. We never overwrite anything in the log, so there's no old value to destroy. A crash can leave the log's tail incomplete, but everything before that tail is untouched and still true.

function set(key, value) {
  const record = { lsn: nextLsn++, op: 'SET', key, value };

  appendToLog(record);   // 1. write the intent to the end of the log
  fsync(logFile);        // 2. force it out of the OS cache onto the disk
  memory.set(key, value) // 3. only NOW change the real data
  return 'committed';    // 4. and only now tell the client "done"
}

Steps 1 and 2 must finish before step 3, and step 3 before step 4. That ordering is the guarantee. Swap any two of them and the whole thing stops working — which is exactly what the crash simulator two sections down will show you.

Recovery is just reading the log back

After a crash, we don't try to repair anything. We reopen the data file, then walk the log from where the data file left off and re-apply every record we find:

function recover() {
  const state = loadDataFile();          // last known-good snapshot
  for (const record of readLog()) {
    if (record.lsn <= state.lastAppliedLsn) continue;  // already in the snapshot
    if (!checksumValid(record)) break;                  // torn tail — stop here
    apply(state, record);
  }
  return state;
}

Watch the log run ahead of the data.

Six operations, broken into the four micro-steps from above. Drag through them and keep an eye on the gap between the log and the data file — that gap is doing all the work.

Notice how long the data file stays empty. For most of the run the only durable record of your committed writes lives in the log — and that's fine, because the log is enough to rebuild everything. Writing to the data file is an optimisation (it keeps recovery short), not the thing that makes your data safe.

Three stores, same workload, one power cut.

The same six operations run in all three columns. Pick a moment, kill the power, and see who comes back intact. Try it at several different ticks — the point isn't any single result, it's which column is boring every single time.

The middle column is the interesting one, because it looks like it's doing the right thing. It has a log. It appends before it applies. It just doesn't wait for the disk — and so it hands out promises the hardware never agreed to. That failure is silent: no error, no corruption, just a handful of confirmed writes that quietly aren't there any more.

Your write didn't reach the disk, and your record might be half a record.

The middle column failed for a specific reason worth understanding, and the left column failed for a different one. Here they are, one at a time.

a. write() returning success is not durability

When you call write(), the kernel copies your bytes into its page cache and returns immediately. That's a deliberate performance decision — it's why writes feel fast. But the data is still in volatile RAM. Only fsync() asks the drive to actually persist it and waits for the answer. Click any stage below to cut the power at that point:

b. A half-written record is not a half-truth — it isn't a record

Appending is safe, but the append itself can still be interrupted. So every log record carries its own length and a checksum of its contents. On recovery we read records until one fails its checksum or runs off the end of the file, and then we stop — everything after that point is discarded, unread.

Here's one record, SET user:1 = 400, laid out as 17 bytes. Drag to choose how many of them made it to disk before the crash:

fsync is slow. That doesn't mean durability has to be.

Waiting for the disk costs roughly a millisecond. Do that once per commit and you're capped at about a thousand commits per second, no matter how big your server is. The fix is to stop doing it once per commit.

If ten clients commit at the same moment, their records are all sitting in the same log, next to each other. One fsync() makes all ten durable. So the database waits a brief window, collects whatever arrives, flushes once, and acknowledges everybody together. Slide the number of concurrent clients:

You have been using write-ahead logs for years.

Append the intent · make it durable · apply it lazily. Once you can see that shape, you start finding it in almost every system that promises not to lose your data.

The mental model, in one breath

1. Never overwrite in place if you can append instead. Appends can be cut short; overwrites destroy the old value on their way to failing.
2. Order is the guarantee. Append → fsync → apply → acknowledge. If you acknowledge before the fsync returns, you are lying to your callers.
3. Recovery is replay, not repair. Which is only correct if replaying a record twice is the same as replaying it once.
4. Every record carries a checksum so recovery can find where the truth stops and discard the rest — writes nobody was ever promised.
5. Amortise the fsync, don't skip it. Batch many commits into one flush; the moment you skip the flush you're back to the middle column.