A website can look finished and still fail when the network drops between two writes. A database can return the wrong row while every page loads. These are ordinary software problems, but screenshot-oriented acceptance gives them very little attention.
Our research into Antithesis and SQLancer was about broadening what a release test can discover.
Two complementary approaches §
Antithesis[1] uses deterministic simulation and property-based exploration to exercise many possible sequences of system events. Its debugging tools[2] let a team replay a failure from a particular state.
SQLancer[3] generates SQL and uses test oracles to expose incorrect database results, including discrepancies between equivalent query forms. It targets errors in the database engine itself, rather than merely checking an application's known queries.
The property is the product requirement §
For a payment workflow, a useful property might be that a retry cannot charge the customer twice. For a queue, a redelivered item must not create two unrelated business records. For an account workspace, a user from one customer must not gain access to another customer's records.
A conventional browser test can exercise the visible workflow. A fault-injection test can challenge its behavior during interruption. A SQL oracle can question the underlying data processing.
Bonfÿre's WorkGraph, queues, data systems and account applications are suitable places to combine these levels. The aim isn't to collect more green test badges. It is to find the conditions under which a real obligation fails.
A screenshot is useful evidence of presentation. The rest of the system needs its own questions.
Sources and further reading
References point to original material linked in the essay, not independently invented citations.
- Antithesisantithesis.com↩
- debugging toolsantithesis.com↩
- SQLancergithub.com↩
Related reading
More articles that explore this subject from another angle.
Independent review is a social condition as much as a software feature: someone must be able to say the plan is wrong.
Read the earlier argument ↗Another essay leads hereThe deployment passed. Why was the website still old?What four public domains and GitHub Pages taught us about deployment destinations and browser readback.
Read the earlier argument ↗Another essay leads herePage three did not mean the same thing twicePagination is a consistency decision about evolving records, not simply an offset and a limit.
Read the earlier argument ↗