Skip to main content

From LinkedIn · · 1 min

Two files are not two sources of evidence

AI can write the code and the tests from the same prompt. That produces two files. Independence comes from a source that can disagree with the implementation.

By Olha Shevchenko. Audits production systems on AWS and Node.js.

AI can write the code and the tests from the same prompt.

That produces two files. It does not produce two sources of evidence.

The code implements one interpretation of the requirement. The test encodes that interpretation. Everything can be green because both missed the same thing. A generated suite will happily cover every line of a function that should never have existed.

This failure is older than AI. Developers have always been able to write code and tests from one mistaken understanding. AI made the duplication cheap.

When I read a test suite during a review, the useful question is not who wrote it.

It is what the test was written against.

A test derived from the implementation can describe what the code does. An acceptance criterion or API contract can define expected behavior. A production incident can define a failure that must not return. Independence comes from a source that can disagree with the implementation.

For a buyer, this changes how much confidence the green suite deserves. If the tests and the code come from one source, the suite does not reduce change risk on its own.

The buyer inherits the cost of finding out which assumptions both files share.