A dashboard shows 94% whether it rests on 12 data points or 2,000.
Checks every AI metric for too little data, outliers and drift before anyone trusts it.
Three independent checks: modified Z-score, IQR cross-check and EWMA completeness.
The problem
Dashboards present metrics with uniform confidence, irrespective of the data volume behind them. A 94% accuracy figure derived from 12 observations is visually indistinguishable from one derived from 2,000 — yet only one merits trust, and most platforms provide no mechanism to distinguish between them.
Who it is for
- Data and ML organizations reporting metrics to executive leadership, requiring differentiation between statistically robust and statistically fragile figures
- Engineering leadership making deployment decisions predicated on evaluation scores
- Organizations previously impacted by a confidently-presented metric subsequently found to be built on insufficient data
Market context
This occupies an adjacent position to the broader AI observability category (Arize, WhyLabs, and Fiddler address model monitoring at scale), yet few platforms isolate statistical trustworthiness of the metric itself — completeness, sample adequacy, outlier influence — as a discrete, first-class evaluation criterion independent of the metric's reported value.
The gap it closes
Established observability platforms report metric values. Few explicitly evaluate whether a given metric merits trust — assessing completeness, sample adequacy, and outlier distribution as independent evaluation criteria, backed by a transparent, auditable methodology rather than an opaque confidence score.
How it works
- MTS is directed at a metrics file or live dashboard data source
- The system evaluates completeness against expected dataset volume, sample size relative to historical baseline, and statistical outlier influence
- A trust classification is applied — supplementing, not replacing, the original metric with essential evaluative context
- Available as both command-line and browser-based implementations, synchronized to guarantee identical results across both
What makes it different
- Explainable by architecture — Modified Z-score with IQR cross-validation, engineered specifically to produce an auditable figure ("4.8 against a 3.5 threshold"), not an opaque model inference
- Singular focus, deliberately scoped — not a general-purpose monitoring platform, but a dedicated instrument for one precise question: metric trustworthiness
- Self-consistent by design — a parity test suite ensures command-line and browser implementations produce identical results, without exception
Architecture
System Overview
Component Breakdown
| Completeness Checker | Compares actual record count against historical median |
| Modified Z-score Engine | Statistical outlier detection, chosen for explainability over IsolationForest |
| IQR Cross-Validator | A point counts as confirmed outlier only when both methods agree |
| Parity Test Suite | Guarantees CLI and browser implementations never silently diverge |
Data Flow
- A metrics file or live dashboard source is provided as input
- Completeness is checked against the historical expected record count
- Modified Z-score identifies statistical outliers
- IQR cross-checks independently — confirmed only if both methods agree
- A trust label and explanation are returned, alongside the original metric
Technology Stack
| CLI | Python | Scriptable, pipeline-friendly |
| Browser version | TypeScript | Same checks, no install required |
| Statistical method | Modified Z-score + IQR | A real, checkable number, not a black-box flag |
Real Design Trade-off, Documented
IsolationForest was considered — it's used elsewhere in this portfolio (SCIP). Rejected here specifically because MTS's whole point is showing your work: Modified Z-score gives a number a teammate can verify by hand.
Performance, Measured Not Estimated
- Parity-tested: CLI and browser versions verified to produce identical results, not merely assumed
- Two-method cross-check measurably reduces false-positive outlier flags versus either method alone
See MTS working
A guided walkthrough, one-on-one. We usually reply within one business day.