← BlogAI tools4 min read

Write the done check before the prompt

An agent loop is only as good as the check that ends it. If you can't write that check, the task isn't ready, for an agent or for a person.

Bottom line

Before you hand work to an agent, write the check that decides whether it’s finished. If you can’t write it, the task isn’t ready. Not for an agent, and not for a person either.

Agents don’t fail at working

The demos all show the same thing: an agent that keeps going. It reads the error, changes the code, runs it again. That part works now, and it will keep getting better.

What breaks is the stop. An agent with a vague goal does one of two things. It stops too early, because “looks good” is easy to reach. Or it never stops, because nothing tells it that it’s done. Both look like model problems. Both are spec problems.

So the question that matters comes before any prompt, and it’s a plain one: how will we know?

Three ways to say “done”

Most definitions of done fall into one of three kinds. Only one of them can end a loop.

A wish

Make checkout faster.

Nobody can check this, so whoever stops first decides it’s done.

A direction

Cut checkout latency, starting with the payment step.

Better. Now you know where to look, but still not when to stop.

A check

p95 for POST /checkout under 300 ms on the staging load test, and every existing test still passes.

A machine can run this. It passes or it doesn’t.

The same task, written three ways. Only the last one can be checked without asking someone.

The step from the second to the third is where the real work is. It forces a conversation nobody wanted to have: which number, measured where, and what we’re allowed to break to get there. That conversation was always needed. Agents just stop you from skipping it.

The loop, with the check in it

Once the check exists, the loop is simple. The agent tries, the check runs, and the result goes back to the agent as the next instruction.

Three rules keep it honest:

  1. Write the check before the prompt. If I write it after, I write it to match whatever the agent produced.

  2. Make the check code, not a judgment. A test, a script, a query with an expected result. If a person has to look to decide, that’s a review. Reviews are fine, but a review can’t end a loop.

  3. Give every loop a cap. A number of attempts after which it stops and hands back, with its log. A loop that can’t succeed should fail cheaply.

What a check looks like

Usually it’s boring. That’s the point.

checks/checkout.done.ts
// Done when: checkout p95 is under 300 ms on the staging load test,
// and the existing suite still passes.
import { runSuite } from './suite';
import { runLoadTest } from './load';
export default async function done() {
const suite = await runSuite();
if (suite.failed > 0) return { pass: false, reason: `${suite.failed} tests failing` };
const { p95 } = await runLoadTest({ route: 'POST /checkout', env: 'staging' });
if (p95 >= 300) return { pass: false, reason: `p95 is ${p95} ms, target is 300` };
return { pass: true };
}

The reason matters as much as the pass. It’s what the agent reads before the next attempt. “Failed” teaches it nothing. “p95 is 410 ms, target is 300” tells it where to look.

Why the cap matters

Here’s a toy version of the loop. Every attempt runs the same check, and the agent usually gets a little closer each time. Run it a few times, then lower the cap.

attempt 1  check: 3 tests failing
attempt 2  check: 1 test failing
attempt 3  check: pass · done

The cap isn’t distrust. It’s a budget. When a loop hits it, that’s information too: either the task is too big for one loop, or the check asks for something the codebase can’t give yet. Both are worth finding out after four attempts rather than forty.

It was never about agents

None of this is new. A good ticket always had acceptance criteria someone could verify. Most tickets didn’t, and teams made up for it with meetings, follow-ups and a senior person who just knew what was meant.1

Agents don’t have that person. They do exactly what the check says, so they expose every place where done was never defined. That’s the useful part. Here’s what the rewrite looks like on three ordinary tasks:

Bug fix

Before: Fix the login bug.

After: A test reproduces the reported login failure and fails on main. The fix makes it pass, and the rest of the suite stays green.

Refactor

Before: Clean up the billing module.

After: No file in billing/ imports from legacy/. Public functions keep their signatures. Every existing test passes unchanged.

Docs

Before: Document the API.

After: Every public route in the OpenAPI spec has a description and one example, and the docs build fails if one is missing.

Three common tasks, before and after.

The teams that get the most out of agents won’t be the ones with the best prompts. They’ll be the ones who learned to say what finished means, in one line a machine can run.

Footnotes

  1. That person is usually the bottleneck nobody names. Writing the check down is also how you stop needing them in every review. ↩

Written by M. Mahbashi, Head of Technology. Replies on X are read.

Keyboard shortcuts
gh
Home
gw
Blog
gp
Work
gl
Lab
gb
Playbook
ga
About
?
This list