Return on Attention: Why AI Code Reviews Are Wearing Us Out

PR volume went up, ticket quality didn't, and the gap got filled with LLMs on both sides of the review: bots reviewing, bots replying, bots occasionally arguing with bots about priorities that only existed in a teammate's head. Our CEO named the actual problem, and it's bigger than code review.

title: Return on Attention: Why AI Code Reviews Are Wearing Us Out published: true description: PR volume went up, ticket quality didn't, and the gap got filled with LLMs on both sides of the review: bots reviewing, bots replying, bots occasionally arguing with bots about priorities that only existed in a teammate's head. Our CEO named the actual problem, and it's bigger than code review. tags: ai, codereview, productivity, softwareengineering canonical_url: https://christine-seeman.com/return-on-attention-ai-code-review/

Our team grew this year, and the PR volume has grown with it. Certainly faster than the ticket generation. New people means more code moving through the pipeline. It doesn't mean more context. A lot of mass PRs needed real thought just to read through: what problem this is solving, why this approach, what actually matters here versus what's incidental. That gap, more code, same amount of shared understanding, is where our troubles started.

When reviews got harder and slower, people reached for the tool that could help them attempt to keep pace. We have an AI code review bot that runs as an automated reviewer on every PR now, and separately, plenty of us run different LLM and coding agents of choice locally to draft review comments. Put those two together and you get exactly what you'd expect: a bot commenting on a PR, another bot replying to it. Some of the LLM created reported bugs, were created in the first place with total confidence by a model that had no way to know it wasn't real. That is a real drawback about the AI coding assistance, confident wrongness.

What it felt like

Comments stopped sounding like the colleagues who supposedly wrote them. Verbose. Every possible reference and citation included, whether the moment called for it or not. We're a team that talks about code plainly, in our own words, and PRs started filling up with AI slop.

One complaint came up more than once: having to leave the PR to get real context, pinging a person or a model outside the thread because the comment itself didn't actually contain enough, or just way, way too much filler. The entire point of a pull request comment is that you shouldn't have to go anywhere else for it.

Another: duplication, and the cost of re-reading. A paragraph explaining code you could read directly in a minute doesn't save you time. It costs you time, on top of whatever the comment was supposed to save you from working out yourself.

Someone on the team put it plainly: not sure what the right balance is now, but the whole paradigm has clearly shifted. Fair. What's less fair is assuming the way we're doing it right now is the only way it could be done.

The scarce resource

A few weeks into this, our CEO said something in Slack that really got to the heart of the whole problem for me. The PR-review complaints were real, but they were a symptom, not the disease. His point: the actual bottleneck isn't PR review specifically, it's that the finite attention of skilled teammates can be easily overwhelmed by LLM-generated content. Human attention is the scarce resource worth guarding most zealously, and inflicting verbose, cheap-to-produce-but-expensive-to-consume writing on each other should be treated as a real anti-pattern. His term for it was Return-on-Attention, ROA: every word you ask someone else to read has to be worth what it costs them to read it.

He backed it with two examples from that same week. Some ADRs had gone out with a lot of repetition in them, and a colleague had to read through all of it. Terrible ROA. A PR that gitignored a single directory, one line of code, ten characters, came with a description running 1,430 characters. Terrible ROA for whoever had to review it.

Neither example is about code quality. Both are about what you're allowed to cost another person's attention to save yourself thirty seconds of editing.

The same idea has a public version. noslopgrenade.com calls it a "slop grenade": pasting a massive AI-generated response into a chat or email where a human would have written one sentence. It's aimed at chat, not code review, but it's the identical failure. An LLM can produce more words than a task needs, for free, and the cost of that surplus doesn't disappear. It just moves to whoever has to read it.

The part I don't love admitting

I noticed the same pull in myself, not as a reviewer but as a thinker. There were stretches where I'd hand a problem to the model before I'd actually sat with it, and I could feel my own workflow discipline getting looser. Not because the answers were wrong. Because I'd stopped doing the part where I figure things out first and ask questions second.

I'm also mid-fight with a specific habit of Claude's: narrating design decisions into comments and code. "We decided this, not that." It shows up unprompted, and it's never once been useful to me. A decision like that belongs in a PR description or a commit message, where it has context and a date and an author. Sitting in a code comment, it's just noise that will be wrong the next time someone changes their mind and nobody remembers to delete it. I'm building a skill specifically to stop Claude from writing that pattern into a codebase.

Where the line actually is

Bot review isn't the problem. Misusing it is.

It's useful when it gives a reviewer an angle they wouldn't have found on their own, or catches a bug, a missed edge case, a real issue sitting in the diff. That's a second pair of eyes, really helping.

It stops being useful the moment it costs the reader more attention than it saves them. Review exists to ship value and to pass knowledge between people. Code quality matters because it serves that, not the other way around. If a comment leaves the code a little better but leaves the reviewer more drained, that's not a trade worth making.

Attributed to you

Here's a rule I use. When posting a review comment, it's attributed to you. Sure, a coding agent might be a co-author…but that isn't who's going to have to follow up. It isn't the one the PR author is going to come to with questions, or pushback. It's you. The PR author has no way to know how it got written. Sure they probably know, especially if you're slapping em dashes all over the place. They're going to read it as your judgment, in your voice, and hold you to it exactly the way they would if you'd typed every word yourself.

So that's the standard before you hit submit: would you have said this, at this length, in this tone? If the answer is no, the fix isn't a better prompt. It's an edit. Leave off what isn't you.