Brackenmere / dev log

Posts Building it with AI agents

Audits that become queues: simplifying what the agents built

Agents add code readily and almost never delete it. Three times in a fortnight I stopped feature work and had the agents audit what they had built, and the lesson was less about the audits than about what happened next: findings only got fixed once they became a queue of small, named jobs.

A03 6 min read

How shall I tell it?

Agents are very good at adding things. They rarely take anything out. After four days of building at full speed, on the morning of 23 September I stopped and asked for an audit.

The first audit

I pasted in an audit prompt I had saved. It is read-only, has a coordinating agent give each subsystem a fresh reviewer, and has a line I now think of as the whole point: "Do not force an abstraction. Prefer boring local code when it is already clear."

Then the coordinator audited its own audit: a coverage pass found three concerns that cut across every subsystem (input, collision, the HUD's sliders) and gave them rows of their own.

Under the hood 22 subsystem rows in the end, 25 accepted findings, and one worker result rejected outright: it described door booleans and line numbers that did not exist in the file. The coordinator caught it by re-reading every cited line.

That stuck with me. One reviewer had invented its evidence, confidently, and the only defence was a coordinator that checked.

The decisions only I could make

The audit ended with questions it would not answer for me. I took most in one go: "In terms of the decisions only I can make, the zoom and weather sliders don't need to be on the results screen. The frozen enemy swing is not intended. A castle prop whose model fails to load should not keep its collider." One I was not sure of: "Just explain to me what I'm deciding here." Then: "Ask me some more questions on all this so we can come up with an implementation plan to clean this up."

Two answers grew the scope. Asked about a slider on narrow screens: "Let's add a minimum width to this that you could possibly work on as well, wherever the landscape pixel width is of a semi-modern phone." The game became landscape only on phones. And: "Also, let's remove all those old screenshots that are sitting there. And the old videos." There were 168 of them tracked in git, 131 MB.

Then I left: "subagent driven, let's go. I am AFK so no approval stops or anything, complete the plan. Anything found during implementation, fix if it's a sub bug that was surfaced, add it into the plan, any questions go with recommendations."

Decisions, recorded

A week later, with several cloud threads running, asking me became the bottleneck. One thread stopped and asked whether to start on its rulings. I could not see what it was waiting for: "What are you waiting on me for? If I answer that, and then you can start implementing those new rulings and fixes, but there's some questions you need from me first." It needed three answers, which I gave in one go.

On 30 September the lore threads went the other way: I told them to decide and write each decision down. By that night the record had passed a hundred rows. "What are decisions 80-100?" A line each came back. Then: "So nothing I need to make a decision on?" Nothing. The rows existed so I could overrule one, not so I had to read them all.

Shaky foundations

By 2 October the project was much bigger, with several long-lived worktrees each running for days. I was worried: "there's a large amount of potential duplication or dead code or unperformant work. Everything seems to be taking longer and longer to do simpler things." And: "I want to make sure that we're not working on shaky foundations. I think a really good audit is needed." Then the line that stopped everything: "Let's not start any new work until we've done a full clean up and restructuring the way we work."

This time there were three audits: dead code, drift (one fact stored in two places, a missing guard) and the same simplification prompt. I wanted action on whatever came back: "probably fixing nearly all of them unless there's some that just are not important. Do not wait for me in any of this. I want you to work long and hard to get us to a real clean starting base."

The results surprised me. The dead-code report called the codebase "unusually clean for dead code". The drift audit found the real problem, a Critical one: a guest who started a New game kept the old game's town. The code was not bloated. It was quietly inconsistent wherever one fact lived in two places.

Under the hood Dead code: 15 findings, none above Medium. Drift: 37, one Critical and four High. Simplification: 26 accepted across 42 subsystems. The deletions that did land took out 143 lines with no change in behaviour.

Clean-up as a queue

What changed was how the findings got fixed: not one big clean-up branch but a queue of small ones, each owned by one short-lived agent and named for what it did: save-session, town-truth, one-walk. Each landed with a line in the handoff.

Some needed me, and again I was a step behind: "Not sure on the chest claimed and what shall we do about movement?" Then, a few minutes later: "how do the town people walk? What was the question?" Once it was explained: "Yes go with A after town-truth lands." The town's people now walk by one set of rules.

Under the hood 17 fix branches over two days, from the dead-code deletions to the castle's torches sharing the kit's light pool. The cost was churn: almost every landing touched the handoff, and several commits exist only to keep it under 10 KB.

The process write-up from that day set a rule: audits that fan out across many reviewers run at most monthly. I broke it three days later.

Four independent auditors

On 5 October I ran out of Claude usage and handed the orchestration to Codex mid-stream: "I've run out of credits. I want you to take over." That afternoon I asked for four separate audits: how the game performs in play, how the agents work ("the quicker we can get to the finish here the better, but I want to ensure we don't introduce bugs"), how we make assets, and the code and assets themselves. "I want each of these to be done independently by independent Astra agents, so they can be thorough and report any findings, where you can orchestrate the fixing of any issues." Astra agents are Codex sub-agents. Each got its own brief, never the others' findings, and none could touch product code. Root merged everything into one numbered queue.

The code auditor found that signing out could silently throw away newer work from another profile. The process auditor found a trap in the way I liked to work: because I kept everything on main and pushed as we went, the tool that picked which tests to run compared against main and saw nothing to test. Work pushed to main had hidden itself from verification. That became a warning in the tools.

What I took from it

  • An audit is only as good as its coordinator. One reviewer will make things up.
  • Ask me for decisions as decisions. Or decide, write it down and let me overrule.
  • Fix as a queue. Small named branches, one owner each.
  • Agents write little dead code and a lot of drift. The bugs were in facts kept twice.

This one is not about the game. It is about how I work with my team of AI helpers, and the day I realised they were brilliant at adding things and almost never took anything away.

Stopping to tidy up

Think of a building crew that works fast and never clears up after itself. After a few days at full speed, I stopped all new work and asked the helpers to inspect what they had built, a bit like a surveyor walking round a house. One helper acted as foreman, handing each part of the game to a fresh pair of eyes and collecting what they found.

The foreman then checked every report. That mattered, because one inspector had simply made up its evidence, quoting details that did not exist. It said them confidently, and the only thing that caught it was a foreman who checked.

Decisions only I can make

Some findings needed me. Should the zoom slider appear on the results screen? (No.) Was an enemy freezing mid-swing on purpose? (No.) A couple of my answers grew the job: the game became landscape-only on phones, and I had them clear out a pile of old screenshots and videos.

Later, with lots of helpers working at once, asking me became the bottleneck. So I flipped it round: decide, write the decision down, and I can overrule any of them. The list soon passed a hundred decisions. I read a handful, and there was nothing I needed to change.

Shaky foundations

A couple of weeks in, everything seemed to take longer and longer. I was worried we were building on shaky foundations, so I said: no new work until we have had a full clean-up.

The results surprised me. There was very little rubbish lying around. The real problem was that some facts were written down in two places, and the two copies drifted apart. The worst example: if you started a new game, you kept the old game's village. That was a proper bug a player would have hit.

A queue of small jobs

What really made the difference was how the fixes got done. Not one huge clean-up, but a queue of small, named jobs, each given to one helper who finished it and moved on. Things only got fixed once they became a list like that.

Later I also had four separate inspectors look at four different things (how smoothly the game runs, how the team works, how the art gets made, and the code itself), each without seeing the others' notes. One found that signing out could lose newer progress. Another found a trap in my own habits that meant some work was quietly skipping its checks.

What I took from it

  • An inspection is only as good as the person checking the inspectors.
  • Ask me for decisions as clear decisions, or decide and let me overrule.
  • Fix things as a queue of small jobs, one owner each.
  • The helpers leave little mess, but they do write things down twice, and that is where the bugs hide.
Building it with AI agents

A08 5 min read

Three AI designers, one brief

When the game felt like an errand chain, I asked three AI designers the same neutral question and kept every other opinion, mine included, away from them until they had answered. How that ran, and what a month of building this way taught me.

Building it with AI agents

A07 6 min read

The team takes shape: root, owners and one reviewer

My kids' playtest notes became four pieces of work, each with its own agent, and one agent above them merging the results. Over four days that settled into a team: a root that orchestrates, owners that build, one reviewer, and a single thing that stays mine.

Building it with AI agents

A05 5 min read

The agents measure themselves

Three times in a fortnight I asked the agents to stop building and measure how they worked: a retrospective from the commit log, a pass through three weeks of their own transcripts, and an overnight round that timed every wait. Each time the numbers corrected something I was sure of.