Arize Integration

Connect Arize to Brief for LLM observability insights. Give Brief visibility into your AI product's traces, evaluations, and engagement metrics.

Last updated: July 5, 2026

Brief + Arize gives your team visibility into how your AI product is actually performing. If you're building with LLMs, Arize (or self-hosted Phoenix) traces every call your product makes. Connect it to Brief so decisions about your AI features are grounded in real usage — not guesswork.

What data does Brief access from Arize?

  • Traces and spans — the calls your AI product makes through its LLM pipeline
  • Sessions — user and session identity carried on those spans, when present
  • Evaluations — pass/fail and scored annotations (correctness, hallucination, and similar checks)
  • Metrics — latency, error rate, and eval-derived quality signals
Brief only requests read access to your Arize or Phoenix project. Brief cannot modify traces, evaluations, or any other data in your account.

From those spans, Brief derives engagement metrics for your product's Product Graph: sessions, unique users, messages per session, return rate, and satisfaction rate (from your evals) — these count only spans that carry user or session identity, or an eval annotation. Error rate and average latency are computed across all spans in the window. If none of your spans carry any identity or eval signal at all — a purely model-monitoring setup — Brief treats the project as not yet reporting engagement data rather than showing misleading numbers.

Why should I connect Arize to Brief?

Ground product decisions in real usage

When your team debates whether an AI feature is working, Brief can reference actual session counts, return rate, and satisfaction — not anecdotes.

Catch quality regressions early

Brief sees your eval pass/fail trends, so a spike in hallucination or a drop in correctness shows up in context, not buried in a dashboard nobody checks.

Works with Arize AX or self-hosted Phoenix

Whether you run Arize's hosted platform or your own Phoenix instance (including inside a VPC), Brief connects the same way.

How do I set up the Arize integration?

Setup time: 2-3 minutes
What you'll need: A Developer API key (Arize AX) or System API key (self-hosted Phoenix)
Auth: Team-level — one connection covers your whole org
  1. Go to Integrations in Brief
  2. Find Arize and click Connect
  3. In Arize AX, create a Developer API key (or, for self-hosted Phoenix, a System API key) — Arize's API key docs walk through this. Read access is sufficient.
  4. Enter your API Key
  5. Optionally enter a Project name — Brief defaults to your first project if you leave this blank
  6. If you're self-hosted or running Phoenix behind a VPC, enter your base URL; leave this blank to use Arize Cloud
  7. Click Save

Because Arize is a team-level connection, one admin sets it up once and the whole org benefits — no per-person authorization needed.

What can I ask Brief with Arize connected?

Once connected, try asking Brief:

  • "How's our return rate trending this month?"
  • "Show me sessions where evals came back negative"
  • "What's our average latency for the assistant pipeline?"
  • "Did our error rate change after last week's prompt update?"
  • "Which sessions have the most turns per user?"

Common Issues

"Connection fails"

Cause: The key type doesn't match the host — a Developer (AX) key won't authenticate against a self-hosted Phoenix instance, and vice versa. Fix: Confirm you're using a Developer API key for Arize AX or a System API key for self-hosted Phoenix, and that the base URL field matches where that key lives.

"No traces found" or metrics look empty

Cause: The project name doesn't match an existing project, or your spans don't carry user.id/session.id attributes or eval annotations yet. Fix: Leave the project field blank to default to your first project, or double-check the name in Arize. Engagement metrics need at least one identity or eval signal per span — pure infrastructure spans won't populate them.

"Self-hosted Phoenix not reachable"

Cause: Your Phoenix instance's base URL isn't publicly reachable, or points somewhere Brief's servers can't resolve. Fix: Use a base URL that's reachable from the public internet (or your VPC's exposed endpoint), and confirm it's the root instance URL, not a specific project or trace link.

"Satisfaction rate seems off"

Cause: Eval label polarity varies by check — for a Hallucination eval, the "good" label might read as factual, not pass. Fix: Brief recognizes common Arize/Phoenix eval labels automatically (pass/fail, correct/incorrect, factual/hallucinated, and similar). An annotation with no label at all falls back to its raw score, but an annotation with a label Brief doesn't recognize is excluded from the satisfaction rate rather than guessed — rename it to a recognized label if you want it counted.

What's Next?

Now that Arize is connected: