Why FlowForge Exists: The Case for Guardrails Around AI Coding

In March 2026, a group of researchers published SlopCodeBench, the first large-scale measurement of what happens to a codebase when an AI agent is sent back to extend its own work, over and over. It is not a benchmark of whether agents can write code. It is a benchmark of what the code looks like after the tenth time they touch it.

The headline finding is narrower than the version of it you have probably heard, and more useful.

Across 36 problems and 196 checkpoints, evaluating 15 coding agents, no agent fully solved any problem end to end. The best agent passed 14.8% of checkpoints. And quality moved in one direction as the checkpoints went on: structural erosion rose in 77% of trajectories, verbosity in 75.5%. Measured against 473 open-source Python repositories, agent code came out 2.3× more verbose and 2.0× more eroded — and the human repositories degraded less often, and by smaller margins, across their own git histories.

So: not every agent, and not every trajectory. Roughly three in four. That is the number worth building against, because it is the one that is true.

The finding that should have been the headline

Buried in the same abstract is a sentence that reads, at first, like bad news for anyone selling a system around AI coding:

Explicit quality guidance reduces initial verbosity and erosion by up to a third, without affecting degradation rates.

Read plainly: telling the agent to write good code works. It starts cleaner — by up to a third. And it decays at exactly the same rate anyway.

That sentence is the reason FlowForge exists, and it is the reason we quote it instead of leaving it out. It says something precise about where the leverage is not. Prompting moves the intercept. It does not touch the slope. You can buy yourself a third of a codebase’s worth of cleanliness at the start and still arrive at the same place, just later.

If your entire strategy for AI code quality is a better system prompt, a stricter CLAUDE.md, or a more detailed style guide, this is the finding that tells you what you have bought: a better starting point on the same curve.

What the paper did not test

Here is the part that matters, and the part we will state as plainly as we can, because it is the whole bet.

SlopCodeBench tested prompt-level guidance. Instructions given to the agent, in language, which the agent is free to follow, deprioritize, or quietly drop twelve checkpoints later when the instruction is no longer in context and the task is hard.

It did not test enforcement. It did not test a commit hook that refuses a function over a complexity ceiling. It did not test a line cap. It did not test mutation-tested static rules, or a review gate that a second agent has to pass before code merges, or persistent memory that carries the architectural decision from session three into session eleven.

Those are different interventions, and the difference is not a matter of degree. A prompt is a request evaluated by the thing being constrained. A hook is a wall. When an agent decides a test is inconvenient, a prompt loses and a pre-commit gate does not.

That is FlowForge’s bet, and the paper is what makes it a bet rather than a slogan: the authors measured the ceiling on instruction and found it, and the intervention class above it is the one nobody had measured.

Why this claim is falsifiable, and what would sink it

We would rather make a claim you can test than one you have to take on faith. So, concretely, here is what would have to be true for us to be right — and what would show we are wrong.

The claim. Under enforced structural constraints, the degradation rate changes — not just the starting point. The slope bends, not merely the intercept.

What would falsify it. Run something like SlopCodeBench’s protocol twice: once with an agent under prompt guidance alone, once with the same agent under enforced gates — complexity ceilings at the hook, coverage floors that block a commit, a reviewer gate on every merge, session state carried forward. Measure erosion and verbosity across checkpoints in both arms. If the enforced arm shows the same slope as the prompted arm — a lower intercept and an identical rate of decay — then enforcement is just expensive prompting, and FlowForge is a wrapper.

We think it bends. We have not run that experiment against this benchmark, and we are not going to imply that we have. What we have is a design that operates at the layer the paper identifies as untested, and production work built through it.

What the system actually does

FlowForge works at three levels, all of them below the agent’s decision-making layer.

Specialist agents. Rather than routing every task to one generalist, work goes to a domain-scoped agent: architecture decisions to one, TDD and coverage to another, threat modeling to another, code review to another. Each carries its own role definition and constraints. The point is not that specialists are smarter — it is that a narrow scope is a smaller surface for the local-but-globally-wrong decision that erosion is made of.

Rules enforced at the hook. Coverage floors, complexity ceilings, review requirements, branch hygiene. These run before a commit lands, not in a post-mortem. A commit that fails does not go through. This is the layer the paper did not test, and it is deliberately placed where an agent’s judgment cannot reach it.

Session persistence. Context loss is the quiet engine of the degradation curve. An agent that does not know the abstraction it is about to collapse was intentional will collapse it, reasonably, every time. FlowForge carries the decisions, the deferred work, and the trade-offs across sessions as state — not as a request to the model to remember.

Time tracking rides along with all of it: every minute maps to a logged action, which is what makes the audit trail and the billing defensible.

What goes wrong without any of it

The failure mode is predictable once you have seen it. First session, the agent is impressive — fast, working code, shipped. Second session, it extends the feature, and with no persistent understanding of the architecture it makes local decisions. It does not know the pattern it is introducing already exists. It adds code that works in isolation and degrades the system in aggregate.

By the tenth session the codebase has absorbed enough of these that it is materially harder to work with than before the AI arrived. That is what “erosion rising in 77% of trajectories” looks like from inside the editor, one reasonable-seeming commit at a time.

What it looks like in practice

One developer used FlowForge to build a medical AI platform across 14 integration domains over four months, producing over 1.52 million lines of code with 80%+ test coverage enforced throughout and a composite quality score averaging 97 out of 100. That is FlowForge’s own record of its own work, and it is measured on our metrics, not the paper’s — a demonstration, not a replication of the experiment above. The methodology behind those figures is on the case study.

The gap

There are AI coding tools. There are quality engineering systems. What has been missing is the layer that connects them — and, per SlopCodeBench, the layer nobody had measured.

FlowForge is a wager on that layer: that the thing which bends the curve is not a better instruction to the agent, but a constraint the agent cannot argue with.

---

Reference: Orlanski, G., Roy, D., Yun, A., Shin, C., Gu, A., Ge, A., Adila, D., Roberts, N., Sala, F., and Albarghouthi, A. (2026). SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks. arXiv:2603.24755. Figures above are quoted from v2 (revised 7 May 2026), the current version of record.

Correction, 2026-09-09: an earlier revision of this post named the wrong institution and the wrong year, claimed every agent tested degraded code without exception, relabelled the verbosity multiple as a duplication rate, and carried two figures that appear nowhere in the paper — one of them a checkpoint solve rate subtracted from 100 and presented as a failure rate. It also omitted the quality-guidance caveat above, which is the finding this post is now built on. Every figure stated here is quoted from the published v2 abstract linked above. The error was ours, not the paper’s.