Is code coverage enough to trust AI-written tests?

No. Coverage records which lines ran during the tests. A test that calls a function and asserts nothing useful still covers every line it runs. A mutation score checks whether a test fails when the code changes, which is what a test is for.

How can an agent use mutation testing?

Run rpr run --diff origin/main --json. Each surviving mutant has a file, line, column, mutator and replacement. The agent writes one focused test per survivor, runs Reapr again, and keeps the test only if the mutant is now killed.

Should an agent change code to kill a surviving mutant?

No. A surviving mutant is evidence about the tests. If a new test shows the code is wrong, that is a bug for a human to decide on. Reapr's mutation-audit skill never changes production code, engine config or thresholds.

Mutation testing for AI-generated code

Updated November 2, 2026. Numbers from the Reapr bench report. Read this guide as Markdown.

Teams now ship a lot of code that an agent wrote, often with tests the same agent wrote. The tests pass and coverage is high. Neither fact says the tests would catch a bug.

Coverage measures execution, not verification

Line coverage records which lines ran while the tests ran. It cannot tell a test that checks the result from one that only calls the function:

it("computes the shipping fee", () => {
  expect(shippingFee({ total: 120 })).toBeDefined();
});

That test covers every line of shippingFee. Change total > 100 to total >= 100, or return 0 instead of the fee, and it still passes.

What a mutation score measures

A mutation engine such as StrykerJS changes the code on purpose, one small change at a time: > becomes >=, a condition becomes true, a return value changes. Each change is a mutant. If a test fails, the mutant is killed. If every test still passes, the mutant survived, and you have a concrete change in behavior that no test catches.

The score is detected / (detected + undetected): killed and timed-out mutants over those plus the survived and uncovered ones. It is the share of the changes your tests would notice.

Why this fits agents

A surviving mutant is a precise, checkable task: this line, this change, no test failed. Reapr prints each one in its JSON output:

{ "file": "src/a.ts", "line": 10, "column": 3, "mutator": "ConditionalExpression",
  "replacement": "false", "status": "Survived" }

An agent can write a test for that exact behavior and prove it worked: run Reapr again and check that the mutant is now killed. The evidence comes from the engine, not from the agent’s own judgment.

The rules that keep it honest

Every shortcut that raises the score can make the tests worse. Reapr’s mutation-audit agent skill writes these down as hard rules:

Never change production code to kill a mutant. If a test shows the code is wrong, report it.
Never edit engine config, thresholds or Reapr flags to change the result, and never create an engine config.
Never delete or weaken tests. Change an existing test only with the human’s OK.
One behavior per test, in the repo’s conventions, inside the diff scope.
Assert intended behavior, not current behavior. When the code disagrees with the docs, types, names or callers, ask the human instead of writing a test that pins a possible bug.
A test is done only when a later Reapr run shows that it kills a mutant. Equivalent mutants, which no test can kill, are listed for the human instead of getting a forced test.

The skill also lists tests that cover the diff but kill no mutant, and then checks the diff for edge cases that mutation operators do not produce: boundaries, empty input, error paths, time zones and numeric precision.

What the skill’s evals showed

The skill was tested on a throwaway TypeScript + Vitest + StrykerJS repo, with and without the skill (pull request #14). On a weak-boundary fixture, the run with the skill took the score from 61.90% to 90.48% and passed 7 of 7 checks. The run without it reached a higher score but rewrote an existing test without asking. With no Stryker config in the repo, the skill stopped and pointed to npx stryker init. Without the skill, the agent wrote its own config and then changed tests.

Try it

The skill has its own page with the install commands and results: reapr.dev/skill.

curl -LsSf https://github.com/reapr-corp/reapr/releases/latest/download/rpr-installer.sh | sh
rpr run --diff origin/main --json

In Claude Code, install the skill with /plugin marketplace add reapr-corp/reapr and /plugin install reapr@reapr, then ask it to check the tests on your branch. For other agents, point them at AGENTS.md.