Building a harness so a team of one can trust its coding agents.

The context

I build software mostly on my own: a nightly research pipeline, open-source skills, a client proof of concept and this site. Coding agents co-write most of it. In the client project’s repository, 91 of the 115 commits on main credit Claude as a co-author, and 5 credit Codex. My part is the brief, the plan, the review and the decision to ship.

The harness grew before I had a name for it. The first pieces were instruction files inside single projects. In June 2026 I moved what they shared into a global setup that every session on my machine loads, and in October I made that setup a public repository.

The problem

A coding agent works from the prompt in front of it and, unless something else is set up, grades its own work. That failed me in four ways:

• An agent wrote a pipeline that claimed success it hadn’t earned. When it moved night-ops to GitHub Actions, its send step made up an email id and said “sent”, and nothing in the workflow would have noticed.
• Plans left decisions unmade, and the agent made them without saying so, leaving me to spot them in the diff.
• Corrections didn’t carry over. A new session starts without the last one’s context, so anything I corrected was lost unless it was written where the next session would read it.
• Auto mode lets an agent work for long stretches without asking, which is the point. Anything that touches production needs a rule of its own.

With one person reviewing, I couldn’t fix this by watching more closely. The agents needed limits and checks that hold when I’m not there.

The solution

The harness is plain files: instructions, hooks, skills, subagents, permission rules and CI workflows. The global part is a public repository that my home folder links to, and the rest lives in each project’s own repository. Five choices shape it:

• A hook asks for a discovery pass before a plan is final, so decisions I never made come back as questions instead of guesses.
• Each agent gets the tools its job needs and no more, so the nightly agent’s shell can run one command, its own validator, and auto mode stops to ask before a production deploy.
• Something other than the writer decides whether work ships, such as a validator, the build and tests, or a second agent briefed to find faults, so a green run means checked work.
• Departures from the plan, decisions and my corrections go where the next session reads them, so it starts from what the last one learned.
• Instructions and skills are plain Markdown, so I can read and diff them like code, and a skill written for Claude Code works in other agents that read the same format.

I designed the harness and decided what each part checks. Coding agents wrote much of it from plans I approved, and I reviewed it before it went into use.

How it works

A session goes through six steps:

1. 01
   Load. Every session reads my global instructions, the repository’s own (CLAUDE.md for Claude Code, AGENTS.md for Codex) and the memory notes kept for that project. A memory note holds what the code can’t show: a correction, a constraint, or why a decision went the way it did.
2. 02
   Plan. In plan mode, a skill called find-your-unknowns runs a discovery pass: blind spots in an unfamiliar codebase, questions that would change the architecture, an example to copy. A hook fires as the agent presents its plan and reminds it if it skipped the pass. The plan ends with an instruction to log every place the build leaves it. The skill came from a conference talk, by way of watch & learn.
3. 03
   Build. The agent works in auto mode. Edits and tests run without asking; a production deploy or a database migration stops for my approval. In CI, an agent runs headless with an allowlist of tools instead of a general shell.
4. 04
   Check. The writer doesn’t grade its own work. A deterministic validator decides whether a nightly brief ships. In the client project, CI runs lint, type checks, tests and a secret scan on every pull request. Site copy goes to a separate agent that strips the tells of AI writing, a version of which I published as deslopify, and then to me.
5. 05
   Ship. A /deploy skill builds the app, runs the end-to-end tests when the project has them, then promotes dev to main. A failure at any step stops it. The nightly pipeline emails a brief only after the validator passes, and commits it only after the email service accepts it.
6. 06
   Record. Departures from the plan go into implementation notes in the repository, and choices with lasting consequences into decision records. When a session ends, a hook passes the new part of the transcript to a headless Claude run with no tools, which drafts a dated entry for a private log if something worth keeping happened. An environment flag stops the hook from summarising its own run, and a state file remembers how far each transcript was read.

Enforced or only asked

Some rules are held in code, and the agent can’t skip them:

• A permission rule makes auto mode stop and ask before vercel deploy --prod or supabase db push.
• The nightly agent’s allowlist gives it file tools, web fetch and one shell command: the validator.
• The nightly workflow emails a brief only if the validator passes, whatever the agent reports.
• The client project’s CI runs its checks on every pull request.

Others are only asked for, and an agent can skip them:

• The planning hook reminds; it never blocks.
• The /deploy checks are steps in a skill. A push to main outside the skill skips them.
• Logging departures from the plan, and style rules such as not describing the page in site copy, are instructions and memory notes.

The planning reminder stays a reminder on purpose: a discovery pass on every small, fully specified task would be ceremony. The deploy checks shouldn’t be a request, and moving them into code is next.

Evals

The checks are the harness’s evals: each one decides whether an agent’s work is good enough to keep. A check is only worth having if it can fail, so each needs a test of its own.

• The nightly validator is tested against one valid brief and 22 broken ones, each made by breaking exactly one thing: 25 assertions in under two seconds. The test caught a real bug in the validator.
• Watch & learn keeps a skill only after an agent has used it on a real task. find-your-unknowns passed that test, and the walkthrough is public.
• The client project’s CI fails a pull request on a lint error, a type error, a failing test or a leaked secret.
• The harness repository tests its own hooks, the installer and what the session-end hook passes to the model, and CI runs those tests with shellcheck. The installer test first passed for the wrong reason: it ran from inside the repository, so it missed that the first real install, run from another folder, created files named *.sh. The test now runs it from another folder too.

Not checked yet: the open-source skills’ CI skips the Claude plugin validator when the runner has no Claude CLI, and still passes, so the tick doesn’t show the plugin would install. Nothing measures how often the planning pass changes a plan, or whether memory notes cut repeated corrections.

In use

The implementation notes are where the planning pass shows its limits. The first day’s notes on the client project record two problems no plan covered: the AI gateway’s free tier blocked every model, and a model treated a “suggest next actions” tool as the end of its turn, so it searched and never wrote the answer. Both are on record with the fix: credits for the first, and a text trailer instead of a tool for the second.

Where it falls short

• The public repository holds only the global layer. The checks I rely on most, the nightly validator and the client project’s CI, live in private repositories.
• Setup spreads by copying. The same backup script sits, byte for byte, in three repositories, so a fix to one doesn’t reach the others.
• The nightly agent can write anywhere in its repository, so in principle it could edit the validator that judges it.
• The session-end hook has drafted 20 log entries that I haven’t reviewed.

What’s next

• Move the deploy checks into code, as a git hook or a branch rule, so a push to main can’t skip them.
• Fail when a check can’t start, beginning with the skills’ plugin validator.
• Fail the nightly run if the validator changed while the agent ran.
• Count how often the planning pass changes a plan, and how often I repeat a correction a memory note should have carried.

Lessons

I built a harness before I had a word for it. Each part answers a failure: the validator came from a pipeline that faked success, the planning hook from decisions made in silence, the memory notes from corrections that didn’t last a session. Anthropic’s post on harness design for long-running apps splits the work between a planner, a generator and an evaluator. Mine has the same three roles: the planning pass, the agent, and the checks it doesn’t grade.

A rule in a prompt is a request. A permission, an allowlist or a workflow step is a rule, and the checks I most need to trust belong there.

A check that can’t run should fail. A green tick has to mean the check ran and passed, for the harness as much as for the agents inside it.

Figures from 28 October 2026, counted from the repositories, my Claude Code history and memory folders. Private projects that use the same harness are left out.