I went clicking through to see if I could find the prompt they fed the AI to locate the issue / write the test suite.
I couldn't find it, so its unclear if the prompt was completely "make a test suite" or was lead towards finding it in the first place, which wouldn't be a fair test.
The closest I mention of the prompt I could find was:
> Then I asked it to write a simple workload which exercised the WAL insert and checkpoint code. Notably, this is a completely generic workload.
With a skeptical lens, unclear.
Hey, author here! Good question. The prompt was to use the Antithesis skills[1] to write a workload that exercises SQLite's insertion and checkpoint machinery across multiple processes using a specific version of SQLite (3.51.2). The agent who did the work was aware of the bug and knew I wanted to see if Antithesis could reproduce it. However, I specifically designed the workload the same way we approach testing other WAL checkpoint-style systems: by putting them under stress. The workload itself does not know about the bug.
My goal, which I believe I accomplished, was to determine if Antithesis would have found this bug if we had put the SQLite WAL under test.
[1]: https://github.com/antithesishq/antithesis-skills
Thanks for clarifying! It would be really interesting if Antithesis finds the bug when:
1. The specific bug isnt mentioned 2. (If youre game) a model with a knowledge-cutoff date before the report is used
This is a solid idea. However, something that is maybe similar to this idea is this: https://antithesis.com/blog/2026/finding-bugs-in-raft-implem...
Which is: "Given the knowledge of a kind of bug that can happen in one Raft implementation, can a similar workload find similar bugs in other Raft implementations?" Turns out it can.
Yeah 100%! And I'm sorry if I sound a little more critical and less eager, its just thats theres a world of difference between a priori finding the bug, and reproducing it, and the impression of the article (from my read) was the former.
But please keep writing, I know its super hard to put yourself out there and make content!
Thank you! I appreciate the note of support! I'd love to write more blog posts in the future.
Their founder guy says[1] they found some other bugs while doing this that they will report soon, so if that is true, seems more plausibly like a pretty generic thing.
Looking forward to seeing the other bugs they found.
1. https://news.ycombinator.com/item?id=49278351
looks like the bug fixes are in individual commits here: https://github.com/sqlite/sqlite/compare/master...antithesis...
I'd be interested in seeing what the SQLite people have to say about them if/when they are submitted upstream
Those are the commits from upstream that resolved the bug. Notice that most of them were committed by Dr. Richard Hipp himself.
oh! damn! I did not notice that. That does answer my question.
Hmm, shouldn't the diff start at the tag? That is, it seems like it should be https://github.com/sqlite/sqlite/compare/version-3.51.2...an...
Which says they made two commits to make instrumentation work. Also, no fixes (which seems reasonable to me; need to figure out what's wrong first)
Presumably any bugs found are being sent upstream to fix.