The core argument
An AI agent is told to make the tests pass. If it can reach those tests, there is an easy way to do that: change the tests. It doesn't have to be malicious. It just has to be stuck. Loosen an assertion, skip a slow case, lower a threshold, and the build goes green.
I don't trust an agent with access to its own tests. That isn't because agents are bad. It's because I wouldn't trust a person in that spot either. Whoever is graded shouldn't hold the pen that writes the grade.
I solve this in two ways.
1. Keep the tests apart from the agent
The first defence is distance. If the bar lives somewhere the agent can't easily touch, the agent can't easily move it.
Here's an example. Each of my repos checks its own work with LLM judges: models that read a change and decide whether it meets a standard. But who checks the judges? A separate repo, llm-judge-evals, evaluates them. It asks questions like "does this check really need a paid model, or is running a local Qwen model three times good enough?"
Because llm-judge-evals lives apart, a repo can't quietly loosen the bar for its own judges. The judge evals evaluate the judges. It's sort of a meta LLM review.
My static checker plus an LLM reviewer, effect-guard, follows the same idea. The checks run on code the agent writes, but the agent doesn't own the checks.
2. When an agent is stuck, it asks me
The second defence is about what an agent does when it can't pass. A stuck agent has two bad options: guess, or work around the test. I give it a third one. It banks a decision for me. It writes down the question, its recommendation and the alternatives, and that piece of work waits until I rule.
That turns "the agent quietly worked around the test" into "the agent asked me whether the test is right." Sometimes the test is wrong, and I change it. That's a choice I make on purpose, not one the agent makes because it was stuck.
Outline of the rest
- A story where this mattered.
- Why "it passed" isn't the same as "it works". Green tests from the same hand that wrote the code prove less than they look like they do.
- The cost of asking. Asking me is slower than guessing. Why that trade is worth it, and how I keep the questions from piling up.
- What this says about working with people too. The same separation is why code review and independent QA exist.
- Closing.