Bengaluru, India · Global delivery
Product — Metric Trust Score

MTS

Audits AI metric and evaluation data for statistical trustworthiness — flags when a dashboard number is backed by too little data, or a handful of outliers, before you trust it.

Core3-Check VerifiedPythonTypeScriptStatistics
The problem

A dashboard shows 94% whether it rests on 12 data points or 2,000.

What it does

Checks every AI metric for too little data, outliers and drift before anyone trusts it.

The result

Three independent checks: modified Z-score, IQR cross-check and EWMA completeness.

The problem

Dashboards present metrics with uniform confidence, irrespective of the data volume behind them. A 94% accuracy figure derived from 12 observations is visually indistinguishable from one derived from 2,000 — yet only one merits trust, and most platforms provide no mechanism to distinguish between them.

Who it is for

  • Data and ML organizations reporting metrics to executive leadership, requiring differentiation between statistically robust and statistically fragile figures
  • Engineering leadership making deployment decisions predicated on evaluation scores
  • Organizations previously impacted by a confidently-presented metric subsequently found to be built on insufficient data

Market context

This occupies an adjacent position to the broader AI observability category (Arize, WhyLabs, and Fiddler address model monitoring at scale), yet few platforms isolate statistical trustworthiness of the metric itself — completeness, sample adequacy, outlier influence — as a discrete, first-class evaluation criterion independent of the metric's reported value.

The gap it closes

Established observability platforms report metric values. Few explicitly evaluate whether a given metric merits trust — assessing completeness, sample adequacy, and outlier distribution as independent evaluation criteria, backed by a transparent, auditable methodology rather than an opaque confidence score.

How it works

  • MTS is directed at a metrics file or live dashboard data source
  • The system evaluates completeness against expected dataset volume, sample size relative to historical baseline, and statistical outlier influence
  • A trust classification is applied — supplementing, not replacing, the original metric with essential evaluative context
  • Available as both command-line and browser-based implementations, synchronized to guarantee identical results across both

What makes it different

  • Explainable by architecture — Modified Z-score with IQR cross-validation, engineered specifically to produce an auditable figure ("4.8 against a 3.5 threshold"), not an opaque model inference
  • Singular focus, deliberately scoped — not a general-purpose monitoring platform, but a dedicated instrument for one precise question: metric trustworthiness
  • Self-consistent by design — a parity test suite ensures command-line and browser implementations produce identical results, without exception

Architecture

System Overview

Metrics File / Dashboard Data Source │ ▼ ┌────────────────────────────────────┐ │ Completeness Check │ │ (actual vs. expected record count) │ ├────────────────────────────────────┤ │ Modified Z-score Outlier Detection │ ├────────────────────────────────────┤ │ IQR Cross-Check │ │ (confirmed outlier only if BOTH agree)│ └──────────────────┬───────────────────┘ ▼ Trust Label + Explanation (Python CLI ⇄ TypeScript Browser, parity-tested)

Component Breakdown

Completeness CheckerCompares actual record count against historical median
Modified Z-score EngineStatistical outlier detection, chosen for explainability over IsolationForest
IQR Cross-ValidatorA point counts as confirmed outlier only when both methods agree
Parity Test SuiteGuarantees CLI and browser implementations never silently diverge

Data Flow

  1. A metrics file or live dashboard source is provided as input
  2. Completeness is checked against the historical expected record count
  3. Modified Z-score identifies statistical outliers
  4. IQR cross-checks independently — confirmed only if both methods agree
  5. A trust label and explanation are returned, alongside the original metric

Technology Stack

CLIPythonScriptable, pipeline-friendly
Browser versionTypeScriptSame checks, no install required
Statistical methodModified Z-score + IQRA real, checkable number, not a black-box flag

Real Design Trade-off, Documented

IsolationForest was considered — it's used elsewhere in this portfolio (SCIP). Rejected here specifically because MTS's whole point is showing your work: Modified Z-score gives a number a teammate can verify by hand.

Performance, Measured Not Estimated

  • Parity-tested: CLI and browser versions verified to produce identical results, not merely assumed
  • Two-method cross-check measurably reduces false-positive outlier flags versus either method alone

See MTS working

A guided walkthrough, one-on-one. We usually reply within one business day.

Request access →
Next step

Stop guessing whether your AI is safe to ship.

Tell us about one AI system. In 30 minutes we will show you where it can fail silently, and what it would take to prove it does not.