Dry Run
Watching an agent long enough to trust it means it was already running unsupervised the whole time you were watching.
The usual advice for trusting a new automated system is to let it run for a while and see what it does. That works fine for something you're watching in real time. It doesn't work for anything meant to run on a schedule without anyone checking each time, because by the time it's run enough for you to trust it, it has already been making those calls unsupervised the entire time you were forming that trust. Proving itself and running unsupervised are the same activity, not two different phases.
Watching isn't the same as checking beforehand
The instinct to "just watch it for a bit" assumes there's a safe window where a new agent's mistakes are cheap to catch. There usually isn't one. A scheduled agent's first real run is functionally identical to its hundredth: same audience, same channel, same consequences if it's wrong. Nothing about being new makes an agent's early output lower-stakes, and nothing about having run a few times without visible complaints proves it's actually been getting things right rather than getting away with being wrong quietly.
The fix isn't watching more carefully. It's separating two different questions that "let it run and see" quietly merges into one: what would this agent actually produce, and should this agent's output go out to anyone. Answering the first doesn't require doing the second at all.
What a dry run actually does
Brief creates every newly created agent paused. When you arm a new agent, the system runs a preview against real data, using the same production executor the scheduled runner will use. The preview produces the actual content the agent generates, but never delivers it. You see what the agent would produce before it goes live.
For a newly created agent, arming runs the preview, then checks whether the agent can reach where it's configured to deliver (a private Slack channel the bot isn't in, or an attempted join that fails, blocks arming). If all checks clear, the agent goes live.
For an already-running agent, editing the identity saves a draft and returns guidance to call the arm tool afterward. When you do, the system runs the preview, grades the draft against the rubric (skipping the delivery check, since nothing about delivery changed on this call), and arms the new identity if it clears.
Editing configuration (like changing the target channel) checks whether the new configuration can deliver before touching the stored config, to leave the currently-running agent untouched if the check fails. If delivery is reachable, the configuration is updated and the agent re-arms on the new config. If the trigger-overlap gate comes back unresolved, the new configuration is saved but the schedule stays paused. Configuration edits do not run a preview; they run only the deliverability and overlap checks.
The preview is different from the post-arm review queue that routes suggestions through judges after the agent is already live. One answers "what will this produce right now"; the other answers "is this suggestion safe to act on" every time the agent makes one.
The honest limit: a preview is not a prophecy
A preview shows what an agent produces right now, against the current data and configuration. If the preview output is empty or execution failed, you can acknowledge that outcome and arm anyway. At a new agent's first arm, if delivery is genuinely unreachable, arming is blocked and you have to fix the problem.
What a preview doesn't show is what happens six months from now when the world your agent operates in has shifted. New signals emerge. Business logic changes. An edge case you didn't write the agent to handle surfaces in production. You can run a live agent on demand (which uses the same executor and real data), but that run will deliver its output. There is no way to see what it would say without saying it. You can edit the agent's identity to surface a new preview before arming that edit, but if the agent is running with no pending changes, there is no preview-on-demand. The honest limit is that the preview is a point-in-time check, not a permanent health certificate.
Where this fits at Brief
The dry-run preview is Brief's answer to the core problem: a scheduled agent's first run carries the same stakes as its hundredth, so watching doesn't reduce risk. Brief lets you answer "does it work" before you answer "should it go live" by running the agent once on real data. You see the output using the exact same executor the scheduled runner uses, decide if it's ready, and arm it or revise it. That's decision capture for agents: making the reasoning and its consequences visible before the commit point, when fixing costs nothing.
Think about the last automated system your team turned on without anyone checking its setup first. Would it have cleared the same bar if someone had actually looked?
Frequently asked questions
What does a dry run show you? What the agent produces when you run it right now, against your current data. When the agent actually runs on schedule later, it will run again with whatever data exists at that time, and produce potentially different output. The preview is a test, not a guarantee of what ships.
What happens if the dry run reveals a problem? If it's the arm-time preview for a new agent, you revise the configuration and re-run the preview to see the change. If it's a preview on an already-live agent's identity edit, you revise the identity draft and call arm again to get a new preview. Configuration edits don't preview-loop; they check delivery once, and you revise or you don't.
Does a successful dry run mean the agent is safe forever? No. The data changes, the world changes, what the agent should do changes. The preview you see when you arm an agent is accurate for that moment. You can surface a new preview by editing the agent's identity, and you'll see what it produces then.
← Back to Blog