Testing against a production-sized SQLite database in CI goes one of two ways: copy the file for every test
(26.2 s for 200 tests on a 125 MB seed), or share one database between tests (flaky, because the state is shared).
forkdb gives each test its own branch instead: the same 200 tests take 2.82 s with nosync.
A fork copies nothing, and a branch stores only the pages it changes.
pip install forkdb
npm install sqlite-forkdb
The npm package is sqlite-forkdb; the PyPI package is forkdb. v0.1.0 • Linux x86_64 • macOS arm64 • Windows x86_64
forkdb loads into the SQLite you already use as an extension. Each host below opens a branch as its own connection.
Your Python's sqlite3 must support loading extensions, and the SQLite behind it must be 3.24 or later. On macOS that means Homebrew's Python (or any Python whose SQLite can load extensions); the macOS system Python cannot.
Tested on Node 24, with node:sqlite and better-sqlite3. better-sqlite3 needs SQLITE_USE_URI=1 set before it is required.
The CLI is for inspecting a store that no application has open: the store file is locked while open. Prebuilt on the v0.1.0 release: Linux x86_64 (glibc 2.28 or later) and macOS arm64 (macOS 11 or later), with a SHA256SUMS file. There is no Windows CLI download.
The extension is also built on its own as forkdb.so, forkdb.dylib or forkdb.dll.
The packages carry prebuilt libraries for Linux x86_64 (glibc 2.28 or later), macOS arm64 and Windows x86_64; elsewhere, build the extension and point FORKDB_EXTENSION_PATH at it.
nosync saves
nosync is opt-in: ask for it when you create the store, with forkdb_init(path, 'main', 'nosync'). A nosync store skips every fsync, so it is for throwaway stores. It survives the death of the process; it is not safe against a power cut.
These are median milliseconds per operation on CI runners, from scripts/bench-nosync.sh: durable first, then nosync.
| Operation | Linux (ext4) | macOS (virtual machine) | Windows (NTFS) |
|---|---|---|---|
| Commit (insert one row) | 0.133 → 0.0152 ms (9x) | 0.827 → 0.0115 ms (72x) | 5.577 → 0.0237 ms (235x) |
| Fork and discard | 0.448 → 0.0190 ms (24x) | 0.304 → 0.0124 ms (25x) | 21.834 → 0.0243 ms (900x) |
| Test fixture (fork, write, commit, close, discard) | 0.680 → 0.1104 ms (6x) | 0.453 → 0.0542 ms (8x) | 27.842 → 0.0962 ms (290x) |
The saving is a property of the disk, not of forkdb: each figure is one disk of one runner on one day. On tmpfs there is no saving, because an fsync costs nothing there.
Run scripts/bench-nosync.sh with FK_BENCH_DIR on the disk you care about.
For a small seed on a disk, a plain file copy per test is as good; branches win on large seeds and on tmpfs.
On a real test suite: in the repository's pytest example, one test's database costs 0.09 ms with nosync (get a private database holding the seed, update ten rows, commit, discard), at every seed size from 1,000 to 1,000,000 rows, because a fork copies no pages.
The same 200 tests (100 reads, 60 committed writes, 40 uncommitted writes) on a 1,000,000-row, 125 MB seed take 2.82 s with forkdb and 26.2 s with a file copy per test; with fsync on, forkdb takes 4.61 s.
One run on one Linux machine, Python 3.12, SQLite 3.45.
What is tested, and against what, is spelled out in the repository's README and KNOWN-LIMITS. The short version:
main and a fork). Their I/O takes turns under one mutex, so it interleaves instead of running in parallel, and one connection cannot write two branches in one transaction. One process per store file: a second process gets an error, not a damaged store.nosync modes. On macOS the durable path calls fsync, not F_FULLFSYNC, so it is weaker against a power cut than that model assumes.sqlite3 can load extensions: Homebrew's works; the macOS system Python and GitHub's actions/setup-python build do not.