The agents measure themselves

On 1 October, with three worktrees and a cloud session all running, I gave one Claude session the job of watching the rest: "I'm worried about coordination and the amount of different things I've got going on. I'd like you to be an orchestrator of the codebase, keeping an eye on things, understanding exactly what's running and what requires what."

By the next lunchtime it felt heavy: "I'm just wondering if we're starting to fill up too much of the context windows of all these separate agents." The art worktree "has been up for two days and it's got all that context mixed in." I wanted to know "where the time bandits are."

A hunch is not a rule

I had a suspect already: "We've got to stop chasing that score of eight when we're not given it... It's costing us a lot of time in the review loops and they're just incrementing very slightly."

Within a minute it had gone out to the agents as my rule. I took it back: "That art review rule was just me hypothesising. I want it to be based on actual reports from agents, and I want you to consolidate that." The new process page kept the lesson: "A rule is not sent to agents as Nick's until he has said it."

What I asked for instead: "taking everything that all these worktrees have learned, how they're working, where the time is, and start to generate a really solid plan on how we work moving forward."

The retrospective

An agent rebuilt the previous three days from commits and evidence, marking each number measured or inferred. My hunch survived, with a twist. Polish rounds on art barely moved the score, half of them downwards, and none of about 25 rounds reached 8. Only rounds that added content made big gains; the reviewers had said so all along: "Density is the cap everywhere." But the one branch that had dropped the bar had also dropped fixing what its review found.

Under the hood Polish rounds moved the mean score by -0.6 to +0.6, against reviewer noise of about 0.5. Content rounds on pieces near 3 gained up to +2.1. Most art rounds took 7 to 20 minutes.

The rest surprised me. Codex plan reviews still found Criticals in their third round. Code re-reviews almost never found anything new. Runtime verification caught seven real defects, all on GPU, performance or deploy paths. The biggest waste was not review at all: three streams had sat idle overnight for about 20 hours, and nothing recorded why. It also found a committed agent transcript with my private instruction file inside. That came out the same afternoon.

One piece, one agent, one sitting

The rules became docs/working.md. Work comes in pieces a fresh agent finishes in one sitting. Art gets one blind review: fix what is broken, no rounds to lift a score, the gallery comes to me.

A token count from the session logs settled the rest. Every tool call re-reads the whole history, so an agent deep in a long session paid, every turn, about what a fresh agent's whole start-up costs.

Under the hood Subagents were two thirds of the spend. Long sessions ran at 300,000 to 575,000 tokens of context a turn, against about 60,000 for a new agent's start. Four streams at once had used 70% of a week's allowance in two days.

So: at most two code pieces at once, and an agent past about 200 turns writes down what it knows and is replaced. I said "Yes adopt the structure", then asked what anyone would: "So do I just start a session in this repo... and start talking?" And: "will it know when it gets to 200 turns?"

Reading their own transcripts

A week later the tests were the problem: "I stopped it running a full 2 hour test suite, we can't let that happen... 2 hours running a test suite is not what we need to do!"

On 10 October I gave Codex a brief pointed at the agents themselves: "Read these logs, understand what went wrong, where I had to steer, what could be changed and improved, where the time bandits were." The aim: "come back with a new orchestrator agent, give it some complex tasks and it not be inhibited by time to complete them all."

Three passes ran in turn. The first found my own complaint in the transcripts, then its cause. The rules already said to run narrow tests; the commands did not. The passes made narrow runs the default, refused a full suite before any work started, and let a green result be reused only when every input matched exactly. No test was deleted to save time.

The third pass corrected the first. An intro piece had run the full check five times in a morning, which looked like waste; traced line by line, its inputs had changed before every run.

Under the hood The first pass scanned 1,835 session files, 7.46 GB of Claude and Codex transcripts. The five checks were about 16 minutes of steps, recorded as "real work but not 950s of proven waste".

Overnight, the agents timed every wait

On the evening of 15 October I set three overnight jobs, the first for the agents: "I want to trim the development time each new job coming up would take." I wanted agents to "do less to complete the same result". The rounds could run side by side: "your job as orchestrator is to ensure they don't do the same work."

A research round measured ten days of sessions first. Waiting and verification took about 60% of an owner's time; the model's own thinking took 21%. Owners had written hundreds of throwaway scripts to drive the browser. And context size barely changed speed: trimming docs was about cost and confusion, not time.

Then three rounds ran at once. One halved npm run check; it also built a warm test server, measured it, found it saved nothing and took it out. One cut what a new agent reads first by 12% and removed every contradiction, proved by a fresh agent answering eight questions from those files alone. One built drive.mjs, a single command that drives the game, and a merge rule for the asset manifest.

Under the hood npm run check went from 213 to 102 seconds; the slowest unit test from 60 seconds to 25 ms. Replaying 16 real merges, plain text merging conflicted in 11 and the new rule in none.

In the morning: "can you give me a rundown of what's changed overnight". My one correction: "drive.mjs is project specific isn't it?"

What I took from it

• A hunch is a hypothesis. Measure it before an agent turns it into a rule.
• Rules in prose are not defaults. The cheap path has to be what the command does.
• Measure the waits. The model's own thinking was only a fifth of the time.

I build Brackenmere with a team of AI helpers. Three times in a fortnight I asked them to stop building and look at how they were working instead. Each time, the real numbers proved something I was sure of wrong.

A hunch is not a rule

By October I had several helpers going at once, and it felt slow. I had a suspect. When a helper made some art, a second one marked it out of ten, and the first kept polishing until it got an eight. I was sure those rounds wasted hours.

Within a minute, the helper in charge had passed that on to everyone as my new rule. I took it back straight away. It was a guess, and I wanted it checked against what had actually happened. Now nothing goes out as "Nick's rule" until I've actually said it.

Looking back

One helper went back over the previous three days. My guess was half right: polishing barely moved the score, and nothing ever reached eight. Only adding more to a scene made a real difference. But the biggest waste wasn't checking at all. Three jobs had sat idle overnight for about twenty hours, and nobody had noted why.

So we made new house rules. Work comes in pieces one fresh helper can finish in one go. Art gets one honest look from a second helper, with no polishing just to chase a score. And a helper that's been going too long writes down what it knows and hands over to a fresh one, like a shift change, because a long-running helper rereads its whole history at every step and gets expensive.

Reading their own diaries

A week later one helper started a full check of the whole game that would have taken two hours. I stopped it, and asked another helper to read three weeks of the others' work diaries and find where things went wrong.

The rules already said to check only the bit you changed. But the easy, default way of running the checks did the lot. So now checking just your bit is the default. Nothing was skipped to save time.

Timing every wait

On 15 October I set them an overnight job: find how a new job could take less time for the same result. First they timed everything. Most of a helper's time went on waiting and checking. Its own thinking was only about a fifth. Then three helpers worked side by side: one made the standard checks twice as fast, one tidied the "read this first" notes, and one built a single simple tool for trying out the game.

What I learned

• A hunch is a guess. Check it before a helper turns it into a rule.
• Writing a rule down isn't enough. The easy way has to be the right way.
• Time the waiting. Most of the time wasn't spent thinking at all.