A FHIR bundle can be structurally valid and still wrong in ways that cost money. The drifted bundle here is valid JSON, valid FHIR R4, and passes a structural validator. It also carries three type-2 diabetes diagnoses written E119 instead of E11.9 — enough to drop 3 patients out of a 46-patient quality-measure denominator without raising anything at all. fhir-gate catches 12 of 12 planted defect types and 0 on the clean bundle.
Both bundles carry identical base data — same 300 patients, same observations, same seed. Every difference below is a planted defect, not sampling noise. The measure: adults with active type-2 diabetes (E11.9) who have a glucose result on file.
| bundle | patients | denominator | numerator | measure |
|---|---|---|---|---|
| clean | 300 | 46 | 26 | 56.5% |
| drifted | 300 | 43 | 24 | 55.8% |
Three patients left the denominator because of a missing decimal point, and two glucose results vanished — one referencing a patient not in the bundle, one labelled with the SNOMED system URI instead of LOINC. Neither run errored. Both produce a number that looks like a measure result.
A glucose result carrying LOINC 2339-0 under http://snomed.info/sct is not rejected by anything — it simply never matches a query filtering on system = 'http://loinc.org'. Nothing errors. The result is just not there. This is the failure mode that takes weeks to find, because every individual record looks fine when you open it.
The unit says inches, the magnitude is still centimetres. Downstream BMI is computed from a height of 170 inches. No validator objects, because in is a perfectly good UCUM unit.
E119 is four characters that a payer's adjudication engine rejects outright. Here it removed three patients from a 46-patient denominator — a 6.5% shift in the cohort, from one missing period.
| # | severity | what it catches |
|---|---|---|
| F2 | critical | duplicate resource id in one bundle — counted twice, or overwritten |
| F4 | critical | Patient with no identifier — a new patient on every import |
| F5 | critical | unit does not match the LOINC code — a height of 170 becomes 170 inches |
| F7 | critical | subject reference points outside the bundle — orphaned from every roll-up |
| F8 | critical | value outside the plausible band — systolic 4200 mm[Hg] moves a cohort mean |
| F9 | critical | LOINC code declared under the SNOMED system URI — invisible, not wrong |
| F12 | critical | ICD-10-CM without its decimal point — the claim is rejected |
| F1 | high | gender outside the value set (M not male) — coerced to unknown |
| F3 | high | birthDate in the future — age-banded measures exclude the patient |
| F6 | high | code and display disagree — humans read one thing, systems compute another |
| F10 | high | dateTime with no timezone offset — a 23:00 result lands on the wrong day |
| F11 | high | status FINAL where the value set is lowercase — treated as missing |
It assumes the bundle already parses and already conforms to R4 — use the HL7 validator for that. These checks sit one layer up, where the resource is valid and the meaning is still wrong.
Terminology is pinned in fhirgate/checks.py rather than fetched, so the repo runs offline and the result is reproducible. In production those tables come from a terminology server; the check logic does not change.
./scripts/run_evidence.sh # everything, into evidence/ python3 -m fhirgate.cli bundles/drifted.json --today 2026-08-04
No dependencies beyond the standard library. Bundles are generated deterministically from seed 20260804.
· Synthetic bundles (300 patients, seed 20260804) — a demonstrator of the method, not a benchmark. · Terminology tables are pinned, not fetched from a terminology server. · Structural R4 conformance is assumed, not tested — that is the HL7 validator's job. · No PHI: every resource is generated, none is derived from a real record.