---
title: Edits an agent shouldn't make one at a time
url: https://www.grahambrooks.com/post/edits-an-agent-should-not-make-one-at-a-time/
date: 2026-11-21T09:00:00-07:00
author: Graham Brooks
description: 'A coding agent pays for a rename per file and per hunk; a refactoring rule costs about the same at five sites as at fifty. Measured on 54 real edits while building colab: 77% fewer tokens than the cheapest hand route, most of the rest spent on output, and one single-site task where editing by hand won.'
tags:
- ai-agents
- refactoring
- developer-tools
- rust
series:
- agents-on-large-codebases
---

# Edits an agent shouldn't make one at a time

*Watch a coding agent rename a function across a repository and you'll see the same three moves on repeat: search, read the file, edit the hunk. The edit is the cheap part. Almost everything else is the agent paying, again, for a change it already understood the first time.*

That pattern follows from the tools. Claude Code's `Edit` replaces an exact string in a file, and it refuses a file the agent hasn't read in the session. So a rename that touches twelve call sites in eight files is a search, eight reads and twelve edits, each with enough surrounding text to be unique. The agent understood the change once. It pays for it per file and per site.

I've spent the last few weeks building the other half of that trade, and measuring it. **colab** is a refactoring tool I've been working on since March 2025: you give it a small script, or a one-line rule, and it rewrites a repository using syntax trees rather than text, across twelve languages plus TOML, JSON and Markdown. The point of this post isn't colab, though. It's the shape of the cost, which I think holds for any tool like it.

## What a rule costs

Here's the rename from colab's eval suite. `config.ParseConfig` becomes `config.LoadConfig`: its declaration, every call, the Go examples in the docs, and the prose that names it.

```sh
colab refactor --write --format json --detail counts \
  -e 'match go::symbol "ParseConfig" and prose and fences { replace "LoadConfig" }' \
  -e 'match markdown::text "Configuration[ParseConfig]" { replace "LoadConfig" }' .
```

That's one tool call. What the agent writes is the rule; what it reads back is a summary — files changed, matches per rule — not the files. The cost is roughly the same whether the symbol appears in three places or three hundred. `Read` and `Edit` are the opposite: nearly all of their cost scales with the number of files and hunks.

So the question isn't whether a rule can be cheaper. It's when, and what eats the difference when it isn't.

## Measured on real edits

I built colab with colab where I could, and logged what it cost. For seven mechanical changes to colab's own code, the colab column below is measured: the bytes of every colab command and its output in the session transcript, including the probes and the attempts that didn't work. The `Read` + `Edit` column is a model, not a second session: a script replays the diff colab produced and prices the cheapest realistic way to make it by hand — one search, one 25-line read per file, one edit per hunk. Tokens are roughly bytes ÷ 4.

| Change | Sites | colab calls | colab bytes | Read + Edit bytes | Saved |
| ------ | ----: | ----------: | ----------: | ----------------: | ----: |
| `FileChange::from_outcome` | 5 | 5 | 7,343 | 10,260 | 29% |
| `conflicts` field in literals | 7 | 4 | 3,800 | 11,013 | 66% |
| `PatternOp` forwarders | 12 | 2 | 2,946 | 18,788 | 85% |
| `SymbolRename::conflicts` | 12 | 2 | 2,938 | 21,032 | 87% |
| `prose_ranges` | 12 | 1 | 1,907 | 18,814 | 90% |
| `run_change_set` callers | 4 | 2 | 1,805 | 7,979 | 78% |
| rename `bound_names` | 2 | 2 | 966 | 3,553 | 73% |
| **Total** | **54** | **18** | **21,705** | **91,439** | **77%** |

Across 54 sites, 77% fewer bytes. The rows say more than the total. The three twelve-site changes saved 85–90%, and the two- and four-site ones saved 73% and 78%, so a handful of sites was already enough. The outlier is the five-site row, at 29%, and the number of sites isn't what made it expensive.

That row shows where a rule's cost actually goes. It wasn't the edit. That change cost 7.3 KB because I previewed a full diff (2.9 KB of output), debugged the pattern by printing its syntax tree (0.9 KB), and needed three rules for three spellings of the same field. The later changes asked for a summary instead of a diff and cost about a quarter as much per site. The lesson I took from it: preview with a search that prints one line per site, apply with a summary, and let the tool's own checks catch mistakes rather than reading every diff back into the context.

The two-site rename went through rust-analyzer's rename, so colab didn't have to find the references, while the hand route still pays to search for them. That's why so few sites still saved 73%.

## Eight tasks, and one loss

The dogfood log is one person's work on one codebase, so I also built a suite of eight tasks an agent is typically given: roll out a minimum toolchain, update docs after a release, migrate Go from `pkg/errors` to `fmt`, rename a module, rename a symbol across code and docs, apply a repository's conventions, move a crate to Rust 2024, and extract a function. Each runs in a temporary git repository and must produce exactly the expected files.

Both sides of this comparison are priced, not observed. colab's side is the commands that work, run for real, with every byte they print counted. It doesn't include the agent finding those commands, which the dogfood log does. The `Read` + `Edit` side comes from the same diff-pricing model as before. On that basis, colab used 8,161 bytes against 29,882: 73% fewer. A time model (2 seconds per tool call, 60 tokens a second written, 2,000 read, plus colab's measured run time) puts it at about 4.5× faster, and I'd treat that as an estimate of the shape rather than a number to quote.

The spread is the same story as the dogfood table. Applying a repository's conventions was one colab call against twelve edits: 89% fewer bytes. Extracting a function was the one loss: 3% *more* bytes and slightly slower. That task touches one loop in one file. By hand it's four calls. With colab it's three — list the language server's actions at that range, apply "extract function", then rename the `fun_name` it generates — and each of those calls has to name the file, the range and the action precisely. When there's one site, reading it and editing it is already about as cheap as it gets.

I left that task in deliberately. A suite where the tool always wins is measuring the suite.

## Is the tool any good?

None of this matters if the rule is slow or wrong. colab's patterns run on ast-grep's matching engine (`ast-grep-core`), under colab's own file walker, prefilter and checks, so ast-grep is the natural comparison. I ran both on the same patterns over 10,025 Rust files (4.1 million lines) copied from a Cargo registry, median of five runs on an M2 Max. One caveat on provenance: these runs are from colab's `main` two days after the v2026.11.15 release, and the performance work they measure isn't in a release yet.

| Task, 10,025 files | colab | ast-grep 0.51.3 |
| ---- | ---: | ---: |
| Search `$X.unwrap()` (24,278 matches) | 1.40 s | 2.04 s |
| Search a `fn … -> Result<$T>` pattern | 1.15 s | 2.06 s |
| Search a pattern found nowhere | 0.24 s | 2.02 s |
| Five rules in one run (56,620 matches) | 2.43 s | 2.48 s |
| Rewrite `.unwrap()` → `.expect(…)`, print the diff | 1.47 s | 2.16 s |

The big gap, about 8×, is the pattern no file contains. colab only parses files that contain the pattern's literal text, so a rule that matches nothing costs a read rather than a parse. A rule that matches nothing is a common agent mistake — a misspelled name, a guessed module path — common enough that colab reports the nearest names it saw when it happens, so that case isn't academic. Elsewhere colab is level or somewhat ahead on this tree, using less CPU, while ast-grep keeps more cores busy. On a small tree (colab's own 238 files) everything takes tens of milliseconds either way; colab is ahead on every search, and ast-grep prints a rewrite diff 7 ms faster.

The more important differences were in the results, not the times. With `if let Some($V) = $E { $$$B }`, ast-grep found 6,171 matches and colab 4,066. ast-grep's default matching ignores trailing children, so it also matched 2,103 `if let … else` blocks, and a rewrite from that pattern would have deleted the `else` branches. With nested calls like `a.unwrap().unwrap()`, ast-grep rewrote the outer one and left 1,417 behind after one pass; colab left none. Neither tool is wrong exactly — they make different choices about what a pattern means. But an agent running a rewrite it hasn't read back wants the choice that can't silently drop code.

## The same shape in documentation

Agents now spend a good share of their edits on Markdown: the plan, the changelog, the spec, the instructions file. The hand route is worse there than in code, because documentation repeats itself. A status column says "Planned" twenty times, so an exact-match `Edit` needs a long, unique anchor, and an agent has to read the file to find one.

colab addresses Markdown by structure instead: a section by its heading path, a table row by its cell values, a list item, a link by its target. The biggest saving was in reading, not writing. Four calls for a document's outline and the sections I needed cost 12.7 KB, against 104 KB to read the four files whole — and that comparison is measured on both sides, since reading a file whole costs its size.

Writing saved less than it did for code: 12% at first, because replacing a section re-sends all of it, then 52% once rows and items could be edited on their own. Half of that second round's bytes went on quoting mistakes in the rule language, and after its error messages improved the next round of doc updates saved 66% with no failed attempts. The part I'd keep even at no saving is the link check: every run refuses to break a link, and its first run on colab's own docs found 11 that were already broken. There's more on documentation in [Give agents addresses, not strings](https://www.grahambrooks.com/post/give-agents-addresses-not-strings/).

## What this doesn't show

Everything above prices the `Read` + `Edit` route from a model of the cheapest way to make the diff by hand. A real agent is usually less efficient than that model: it reads more than 25 lines, re-reads files after editing them, and fixes the hunk it got wrong. So I'd expect real sessions to favour the rule more, not less. But that's an expectation, not a measurement. Running real agents on the eight tasks, colab against `Edit`, is the next step, and until it's done the honest claim is about the shape of the cost rather than its size.

It's also one person's codebase, mostly Rust, on one machine. The dogfood rows are changes I chose to make with colab, which biases them towards changes a rule is good at.

## The shape is the point

The finding I'm most confident of is the shape, because it doesn't depend on any of the constants: an agent's hand route costs per file and per site, and a rule's cost is mostly fixed. In what I measured, two or more alike sites were enough for the rule to win when its output stayed lean, the twelve-site changes saved 85–90%, and nothing about the mechanism suggests the saving shrinks at a hundred. What ate the saving wasn't too few sites but too much output: full diffs read back into the context, and patterns debugged by trial. The one change where hand editing won had a single site.

That's an argument for giving agents tools whose cost is per *change*, not per *site*, for anything mechanical: renames, migrations, convention rollouts, the docs that name the thing you renamed. It's not an argument against `Edit`, which is still the right tool for the one-off change in one place. The agent's job is to tell the two apart, and a summary of "12 sites in 8 files" from a dry run is a cheap way to find out which one it's looking at.

colab is [open source](https://github.com/grahambrooks/colab), and the eval suite, the dogfood log and the ast-grep comparison are all in the repository with their method, so you can redo the sums with your own constants. What changed in the latest release is in [colab v2026.11.15](https://www.grahambrooks.com/post/colab-2026-11-15-agent-refactoring/).
