Tests as Data: Ensuring AI-Generated Code Does What We Want
Martin Fowler, Wednesday, November 11th, 2026
When AI agents write both code and tests, moving the human-AI contract into readable test data restores trust in the tests.
Matteo Vaccari of Thoughtworks argues that automated testing loses its assurance when coding agents write both production and test code; developers he interviewed said they no longer fully trust their tests.
He makes three claims: when AI writes the code, developers become its customers; customer tests, not developer tests, become the primary contract; and putting that contract in test data instead of test code protects it from unexpected changes by agents.
AI makes the needed DSL parsers and custom test runners cheap to build.
The approach draws on Extreme Programming acceptance tests, Fit and specification by example, Go's tabular tests and approval testing's readable, diffable output.