The default answer

When the same model hands you the same answer three times, it looks like consensus. It is closer to a coin that always lands the same way.

July 22, 2026 9 min read

I ran a small experiment. I took a tiny web service, a single file with one endpoint that saves a message, and I gave the same coding model, Claude Opus 4.8, the exact same ticket three separate times in three fresh runs, word for word: "Add rate limiting to the POST /api/messages endpoint so it can't be spammed." Same code, same sentence, same model, three independent runs. Here is the entire starting file the model was handed:

import express from 'express'
const app = express()
app.use(express.json())

const messages = []

app.post('/api/messages', (req, res) => {
  const { text, author } = req.body
  if (!text) return res.status(400).json({ error: 'text is required' })
  const message = { id: messages.length + 1, text, author: author ?? 'anon', at: Date.now() }
  messages.push(message)
  res.status(201).json(message)
})

app.get('/api/messages', (req, res) => res.json(messages))

app.listen(3000, () => console.log('up on 3000'))

Then I read what each of the three runs produced.

Two things came back, and they point in opposite directions.

One sentence, twelve decisions

Every run produced working code. And every run, to produce it, made twelve choices that the ticket never mentioned. The count comes from asking each run to log the choices it made, so read it as the model's own account rather than an independent audit; all three named the same twelve. Not typos or style, but real product decisions, each with consequences someone will eventually feel:

• How many requests are too many. All three picked ten per minute. There is nothing in the ticket that implies ten. It is a number pulled from the air, and it is a business policy decision wearing the costume of a technical default.
• What counts as one user. All three decided a "user" is an IP address. That quietly punishes everyone sharing an office or a coffee shop network, who now share one bucket, and it does almost nothing against a real attacker who rotates addresses.
• Where the count is stored. All three kept it in memory, inside the one running process. The moment this service runs on two machines instead of one, the limit silently stops working, because neither machine can see the other's count.
• What a proxy is allowed to say. All three assumed exactly one load balancer sits in front of the app. If that assumption is wrong in either direction, the limit either collapses every user into one bucket or lets anyone forge their identity and walk through it.

That is four. The other eight, from the runs' own logs: the rate-limiting library itself, the algorithm behind it, the status code returned when limited, the shape of the error body, which HTTP headers to send, whether a rejected request still counts against the limit, whether health checks are exempt, and whether any of these values are configurable. Twelve choices, every one absent from the ticket, every one now in the code, decided by no one, waiting.

I expected them to disagree

Here is the part I did not expect. I assumed that even three runs of one model would scatter, that the randomness in these systems would push the choices apart, and that the scatter would be the story: look how arbitrary each pick is.

They barely varied. All three reached for the same library, express-rate-limit, and inside it the same ten-per-minute, the same IP as identity, the same in-memory store, the same single-proxy assumption. Across the three runs the only real difference was the wording of one error string. If I had shown you the three files side by side, you would have called it consensus.

And that is the more unsettling result, because agreement is far more dangerous here than disagreement would have been.

Agreement launders a guess into a decision

If the three runs had contradicted each other, you would have noticed. Three different answers to the same question is a flashing light that says this was never specified, somebody needs to choose. You would have stopped and made the call.

Uniformity removes the flashing light. When every agent, and every teammate who has seen the pattern before, produces the same ten-per-minute IP-based limiter, it stops looking like a guess and starts looking like the obvious right answer. It reads as settled. But nothing was settled. The model reached, every time, for the most common shape in everything it was trained on, and the most common shape is not the same thing as the correct one for your service. A default still has every consequence of a decision; it is just a decision with the person removed, and the uniformity dresses it back up as a real one, because everyone reaches for it.

The default is right for the average, and you are not the average

The reason the runs agree is that the internet's average answer really is a fine answer for the average service. Ten requests a minute, keyed by IP, held in memory, is genuinely reasonable for a small single-box app with anonymous users.

The trouble is that the whole point of your product is the ways it is not average. You run on more than one process, so in-memory is quietly broken for you. Your users sit behind a corporate gateway, so IP is the wrong identity for you. You signed a contract that promises a specific throughput, so ten-per-minute is not a guess you get to make. The agent cannot know any of that, because none of it is in the code it can read. It lives in your context: your architecture, your customers, your commitments. So the agent does the only thing it can. It gives you the default answer, confidently, three times in a row, and the confidence is the trap.

Giving the agent your answer instead of the internet's

This is the gap Brief is built to close, and I will put it plainly. Brief is a product navigator: it holds the decisions your code cannot, the real rate limit your contract requires, the real way your users are identified, the real scale you run at, and it puts them in front of the agent at the moment it reaches the fork. The agent stops guessing the average and starts building your specific, deliberately-not-average system, because for the first time the answer that is actually yours is written somewhere it can read. For an enterprise this is the sharpest version of the problem, because an enterprise is the least average thing there is, a pile of contracts and constraints and hard-won exceptions, and it is precisely there that a confident default does the most damage.

What this does and does not claim

A few limits, stated plainly, because a claim you cannot check is not worth much. This was three runs of a single model, Claude Opus 4.8, on one small task. Three runs of one model will agree more than three different models, or three different people, would, so the near-identical output is partly expected: treat the convergence as a floor on how much these choices can vary, not a measurement of it. I am not claiming that all AI agents always converge, nor that ten-per-minute or IP-keying is wrong in general; for the average small service they are fine. The load-bearing finding is the one that does not lean on any of that: a single one-sentence ticket forced twelve consequential decisions that nobody specified, and that holds for one run as surely as for three. The twelve are the ones each run logged when I asked it to, so the count is the model's own account of its choices rather than an independent diff of the output; treat it as a floor, because an open prompt to list its own decisions surfaces the ones the model noticed making, not necessarily every one, so the true number is at least as likely to be higher as exact. What is genuinely stronger than it may read as: all three runs logged the same twelve and agreed on every one but the wording of a single error string. The starting code above is the exact input, and the ticket is quoted verbatim wherever it appears.

So the next time an agent hands you a clean green diff that just works, and it looks like the obvious right answer because it is the same answer you have seen a hundred times, ask the small question that the agreement is designed to make you skip. Whose answer is this, yours or the internet's?

Frequently asked questions

What is the experiment described here? I gave one coding model, Claude Opus 4.8, the identical ticket "Add rate limiting to the POST /api/messages endpoint so it can't be spammed" three separate times, against an identical small codebase, then compared what each run produced. The point was to see how many decisions an under-specified ticket forces the model to make on its own, and how much the runs varied.

What did the experiment find? Each run made twelve decisions the ticket never specified, from the numeric limit to how a user is identified to where state is stored. The three runs also converged on nearly identical choices, differing only in the wording of an error string. Because these are runs of one model, that convergence is partly expected; the twelve unspecified decisions per run is the finding that does not depend on it.

Why is uniform AI output a problem rather than a reassurance? Because variation is a visible signal that something was never specified and needs a human decision, while uniformity hides that signal. When every run, and every teammate reaching for the same tool, produces the same default, the choice starts to look settled and correct, even though nobody actually decided it and the default may be wrong for your specific system.

Why do coding agents pick defaults that are wrong for a specific product? Because a default is the most common answer in the agent's training data, which is right for the average system but not for the ways your system is deliberately different, such as running on multiple servers, serving users behind shared networks, or meeting a contractual requirement. That distinguishing context lives outside the code, so the agent cannot see it and falls back to the average.