Coding agents in production: what an agentic harness does

How an agent works with a frame

A developer hands over a ticket: a new export in CSV format. The agent first reads the project rules in the repository. It finds the existing export service and extends it instead of building a second one. It writes the tests, runs the build, linter and test suite, fixes two failures itself and submits the change for review.

The reviewer then checks only what machines cannot judge: does the change solve the right problem, and does it fit the direction of the product? That is the goal. Whether a team reaches it is decided not by the model but by the frame the agent works in.

Why many teams do not see this effect

The same agent without a frame works with what it happens to see: the task, the open files and its general knowledge. It does not know the project's conventions, the architecture decisions of past years or the reasons behind them. The result is code that compiles and looks plausible but sits beside the existing structure.

That code ends up in review, where it costs more time than obviously broken code. If the agent produces in ten minutes what takes two hours to review, the bottleneck has merely moved.

Two studies document this pattern. In 2025, METR observed experienced open-source developers in their own repositories: with AI tools they took 19% longer, yet believed they were faster. The DORA 2024 report found lower delivery stability and throughput in organisations with higher AI adoption, alongside higher perceived productivity. In both cases, developers used the tools individually, without shared rules and without measurement.

So the studies do not say that coding agents are useless. They say the benefit is not automatic. Whoever wants it builds the frame and proves the result.

The five parts of the harness

The term comes from test automation: a harness holds the component under test, feeds it inputs and observes its behaviour. Anthropic's guide to building agents describes the same experience: the most successful implementations relied on simple, transparent patterns and carefully designed tool interfaces, not on complex frameworks. The effort goes into the frame, not the model.

Applied to coding agents, that frame has five parts:

• Project rules: instruction files in the repository describe the architecture, conventions, preferred libraries and patterns to avoid. The agent reads them at the start of every task, like a new colleague reading the developer documentation.
• Tool access: the agent runs the build, tests and linters itself and gets their output back. Without that access, it guesses whether its code works.
• Guardrails: they define which files the agent may change and which commands it may run. Before migrations or interface changes, a human signs off.
• Feedback: tests and CI deliver a verdict that does not depend on the agent's self-assessment. A failing test means: try again before a person sees the change.
• Metrics: what counts as success is agreed up front. The final assessment then does not rest on impressions.

Tests and CI: the strongest lever

An agent that runs its change against the test suite fixes a large share of its own mistakes. For that, tests need to be fast enough to run several times per task and meaningful enough to catch real defects. A codebase with few or slow tests is therefore a difficult starting point. Unless writing tests is the first use case.

CI is the second instance the agent cannot bypass. What it misses locally, the pipeline catches. Only when both stages are green is a human review worth the time.

Measure what matters

Generated lines of code say nothing about value. More code means more maintenance, and an agent produces code effortlessly. Developer satisfaction is a weak indicator too, as both studies show.

Useful criteria refer to the delivery process. Most teams can derive them from Git and their ticketing system. What matters is fixing them in advance: whoever decides at the end what success means will always find a number that fits.

• Lead time: how long does a task take from start to merge, compared with similar tasks without the agent?
• Review time: how much time do reviewers spend on agent-generated changes, and how many correction rounds are needed?
• Defect rate: how often do these changes lead to rework, reverted merges or defects in production?
• Share of tasks completed independently: which tasks does the agent finish in the agreed quality without manual correction?

Protecting code and data

A coding agent sends parts of the source code to a model. Whether and where that is permissible is a matter for contracts and your data protection officer. The technical side belongs in the harness: a gateway such as LiteLLM funnels all model calls through one place and defines which models are allowed, which provider handles which task and what gets logged.

If the code must not leave your own network, open models on your own infrastructure are an option, for example with Ollama or vLLM. They are usually less capable than the large cloud models, but often sufficient for clearly scoped tasks in a good harness.

How to start

Licences for the whole team are the wrong first step. They create usage but no frame, and everyone ends up working differently. More reliable is one use case in the real codebase: a task with recurring effort, a verifiable result and manageable risk.

For that use case you build the harness, the agent works in the repository, and the team evaluates the results against the agreed criteria. After that you know whether a rollout is worthwhile, with which guardrails and which tooling. If it is not, that is a result too, and a cheaper one than a failed rollout.