Held-out evidence
Triage benchmark, reproduced from the repository
This 30-case set is the development regression suite: the rules were tuned against development cases and this held-out split tracks them. A larger, freshly blinded corpus is the next planned measurement — treat these as engineering numbers, not field results.
The measured results below come from npm run evaluate:local on the versioned synthetic corpus in evaluation/cases.json. Explanatory copy and the Wilson interval are reported separately. Local engine timing excludes voice, model, geocoding, and routing latency and varies by machine.
Independent synthetic-data demonstration; not an official 112, ERSS, government, or C-DAC service.
Results
Location accuracy checks expected text terms, not coordinate accuracy or successful geocoding. Threat detection checks the three labeled threat cases; it is not general threat recall.
| Metric | Value | Denominator |
|---|---|---|
| Critical recall | 100% | 9/9 critical cases retained — Wilson 95% lower bound 0.70 |
| Type accuracy | 60% | 30 held-out synthetic calls, corpus v1.0.0 |
| Severity accuracy | 60% | 30 held-out synthetic calls |
| Under-triage | 23.3% | 7/30 calls graded below the reference severity |
| Over-triage | 16.7% | 5/30 calls graded above the reference severity |
| Location accuracy | 100% | 25 cases with a reference location |
| Threat detection | 100% | 3 cases with weapon or entrapment threats |
Context: human and protocol baselines
Published US field-triage guidance targets under-triage at 5% or less while accepting 25–35% over-triage; a systematic review observed real-world ranges of 1.6–72% under-triage and 9.9–87.4% over-triage. A Copenhagen study of 108,607 emergency calls found human dispatchers recognized out-of-hospital cardiac arrest with 72.5% sensitivity. Kwik 112's 30-case held-out result is reported directly and does not claim clinical equivalence from a small synthetic corpus.
Reproduce it yourself
git clone https://github.com/areycruzer/kwik-112
cd kwik-112
npm install
npm run evaluate:local # regenerates every number aboveRaw machine-readable output: local-held_out-latest.json (committed to the repository).