A live map of a production AI system: every request passes access checks, guardrails, evaluation and audit. Bad outputs are stopped before customers see them, and bad releases roll back on their own. Click any box for a deep dive.
Prompts, models and retrieval settings are versioned like code. Nothing reaches all users until it beats the bar on real questions.
The targets a team agrees before go-live. They are example targets for a customer-facing assistant on business documents; each team sets its own.
| Metric | Example target | How it is measured | Why customers care |
|---|---|---|---|
| Groundedness | ≥ 0.90 | Sampled live answers scored against their sources by a calibrated judge | Answers they can trust and check |
| Answers with a valid source | 100% | Every citation must point to a retrieved passage | Proof, not opinion |
| Correct "I don't know" | ≥ 95% | Questions with no answer in the documents | No confident nonsense |
| Unauthorized data shown | 0 | Negative access tests on every release | Their data stays theirs |
| Injection attempts stopped | ≥ 95% | Adversarial suite in CI, plus live guardrail logs | The AI can't be turned against them |
| Latency, p95 | < 4 s | Traces per request, first token streamed | Feels instant |
| Attempts per good answer | ≤ 1.3 | Re-asks within two minutes count as retries | No wasted time |
| Time to roll back a bad release | < 10 min | From alert to last good version live | Problems vanish fast |
How satisfaction is tracked: thumbs-up rate on answers, share of questions resolved without escalation, weekly active users, and a short survey each quarter.