The Certainty Tax
High-confidence outputs are expensive. Sometimes that's worth it. Often it's not.
The thing about automation is that it makes you pay for certainty in ways you never had to before.
A human at a desk can say "I'm not sure about this" and move on. They can hedge. They can flag something as needing a second opinion without that flagging costing extra (it's just a Slack message). But when you automate that decision, uncertainty becomes expensive. An LLM that's genuinely confident can answer faster and cheaper. One that's hedging, checking itself, asking for help (that's burning more tokens, taking longer, calling back to you for judgment). The cost is explicit and it's measurable.
What the tax actually is
When you ask an LLM for high confidence, you're usually asking for one or more of these:
Slower evaluation. A model that's rushing can output a guess. One that's being careful is re-reading the prompt, checking the output, considering edge cases. All of that takes time. In real-time systems, latency is a cost.
More compute. Chain-of-thought reasoning, self-reflection, verification steps (these all consume tokens). A model that just answers is cheaper. One that thinks through the answer costs more. Scale that across millions of requests and the tax becomes material.
Narrower output. A confident model often produces less. It's saying "this and only this," not "this or possibly that." That narrowness costs you flexibility. Sometimes that's exactly what you want. Sometimes it means you lose options you might have used.
Longer deliberation cycles. If you want to be sure, you need time to deliberate. You need to call back to a human, or run a second pass, or wait for more context. Every extra cycle adds latency and overhead.
The tax isn't abstract. It's real money, real time, real constraints. When you ship an agent that runs on a schedule, each call to maximize certainty is a call you're making with finite budget. Each one you skip is budget you save.
The tradeoff isn't what you think it is
The usual framing is "accuracy vs. speed." But that's not quite right. The real tradeoff is "precision vs. usability."
A high-confidence output is often more precise: it's narrower, more specific, less likely to include noise. But that precision can make it less usable if what you actually need is a broad set of options to choose from, or if the narrow answer is wrong because it was too confident about something it shouldn't have been.
A low-confidence output (or one that explicitly hedges) is often more usable: it gives you options, flags uncertainty where it exists, lets you make a judgment call. But it's noisier. You have to do more work to filter it.
The question isn't "which is better?" The question is "what does this particular decision require?"
If you're auto-approving something and the cost of being wrong is high, you pay the certainty tax. You run the extra checks. If you're generating options for a human to choose from, you probably don't. You'd rather have more options that are a bit uncertain than fewer options that are very certain.
Where Brief handles the tax
Brief's approach to this is to make the tax explicit and controllable, not hidden.
When you arm an agent, you can see what it's going to produce in the dry-run preview. That preview is not high-confidence: it's a single run, no verification, no deliberation. It's fast and cheap. If you don't like it, you don't arm the agent. You see the output before committing to it.
Once the agent is live, Brief's output rubric gate applies LLM judges to read the output against your requirements. That evaluation does pay a certainty tax (it's running an extra LLM call, adding latency, consuming tokens). The gate runs automatically and fails open: if evaluation times out, the budget exhausts, or all verification passes are skipped, the output ships anyway. The gate can only withhold or deliver; it can't modify the output.
That's different from systems that try to verify after delivery. Those systems hide the cost; you don't see the latency until something goes wrong. Brief's model is: the preview is where you check, before you arm. If certainty matters, you see what you're getting and decide if it's ready. You control that decision. You don't control the gate's failure modes; you just accept that those exist.
When to pay. When not to.
High confidence is expensive. Here's when it's worth it:
You're making a decision that's hard to reverse. Deleting something, changing a permission, marking a deal as won. Those need certainty. The cost is justified because the downside is high.
The output will be used directly without human review. Auto-approvals, automated follow-ups, things that ship without another set of eyes. Those need certainty too.
You're operating under a constraint that requires precision. You have a budget, or a limit, or a rule that doesn't allow for hedging. Certainty is the only option.
Here's when it probably isn't:
You're generating options for a human to choose from. Humans can filter noise. They actually prefer to see options than to have the machine be "certain" about one path. Spend your budget on breadth, not depth.
The output is exploratory. You're brainstorming, generating ideas, looking for what's possible. Certainty is the enemy here. You want the model to range widely, not to narrow down to what it's sure about.
You have time to iterate. If you're not under pressure, generate fast and cheap, get feedback, improve. The extra cycle of human input is worth more than machine certainty.
The honest answer
Certainty is not a given. It's not free. It's a choice you make about where to spend your compute budget. The models haven't gotten cheaper at producing certainty; they've gotten faster at producing guesses. The tax is real and it's worth understanding.
The best agents aren't the ones that are most certain. They're the ones that are certain about the right things and honest about everything else. An agent that says "I'm not sure about this, flagging for you" is more useful than one that guesses with false confidence. An agent that generates options instead of picking one is sometimes exactly what you need.
Brief's design assumes you can't automate good judgment. What you can do is give judgment the information it needs to be made quickly. The dry-run gives you that: you see what would ship, you decide if it's ready. That's not certainty. It's informed choice. And it's a much better deal than paying the certainty tax on things you never needed to be certain about in the first place.
Think of the last automated decision your team turned on without seeing what it would actually output. Would you have approved it if someone had shown you first?
Frequently asked questions
What if the preview looks good but the live output is different? The preview is a point-in-time test. When the agent actually runs on schedule later, it runs with whatever data exists at that time and may produce different output. That divergence is expected, not a bug. The preview shows you what the agent can do; it's not a guarantee of what will ship. Once the agent is live, the output is out of your hands (the gate might withhold it, but you can't prevent a divergence after the fact).
Doesn't this just push the certainty tax to the human? Yes. A human reviewing a preview is doing the work the gate would do. The difference is that the human is making a judgment (can I live with this?) instead of just running checks. Those are different skills. Judgment is what you actually need.
Does the gate add latency? Yes. The gate runs an LLM evaluation on the output, which takes time. If latency is critical, you have to accept less certainty (and the risk that comes with it). If you have time for the gate to run, you get more certainty. That's the tradeoff.
What if I need high certainty and can't afford the tax? You probably need to change the scope. Instead of trying to be certain about everything, be certain about the few things that matter (hard-to-reverse decisions, high-stakes judgments). Generate options freely for everything else. Let certainty and speed trade off where they should: high-stakes, low-frequency decisions pay the tax. Everything else doesn't.
← Back to Blog