Data engineering · compliance evidence

soc2-evidence

An evidence completeness gate for SOC 2 / ISO 27001 control sets. It catches the pack where the readiness tracker reads 12 of 12 and every control has an attachment — while the artifacts are stale, cover the wrong population, prove the wrong thing, or are flatly contradicted by the access log. Offline, no keys, standard library only.

Public · synthetic demo
Scope — this is not an audit

The author is an evidence engineer, not an auditor. This tool automates evidence collection and completeness checking. It issues no opinion, certifies nothing, and does not substitute for a licensed CPA firm's SOC 2 examination or an ISO 27001 certification body's audit. Its findings are inputs for that work, produced before fieldwork.

The problem

"Every control has evidence" is not "every control is evidenced."

The tracker counts attachments: 12 controls, 24 artifacts, 12 green ticks, book the auditor. Nobody opens the files. Underneath, the quarterly access review was pulled from the HR system so contractors and service accounts were never in it; the backup control has a year of backup-succeeded logs and zero restore tests; four controls are evidenced by artifacts collected months outside their own refresh window; and five people who left still hold production grants. Each one is a finding raised in week one of fieldwork, and the check most teams ship — does this control have at least one attached artifact? — waves all four through.

The money shot

The same evidence pack, checked two ways

① Tracker PASS vs gate BLOCKED
checkverdict on the shipped pack
naive has an artifact?12/12 evidenced, 0 gaps → PASS (books the audit)
soc2-evidence gate14 findings across 7 of 12 controls → BLOCKED
  soc2-evidence · the readiness tracker says the pack is ready
  ==================================================================
  clean pack     · gate findings:   0                        → PASS ✅
  shipped pack   · tracker 'has an artifact?': 12/12 evidenced, 0 gaps → PASS (books the audit) ❌
  ------------------------------------------------------------------
  evidence gate on the SAME pack:
    findings: 14  by kind: {'stale': 4, 'population': 1, 'artifact_type': 1, 'contradiction': 8}
    controls still clean: 5/12
  ------------------------------------------------------------------
  smoking gun 1 · RM-01: evidence is 501d old against a 365d window (136d past due)
  smoking gun 2 · AC-03: review covers 34/53 of the roster — never in scope: contractor x11, service_account x8
  smoking gun 3 · BC-01: 2x backup_job_log attached, 0x restore_test_report — the control requires the restore test
  smoking gun 4 · 8 live grants held by 5 terminated identities, e.g. cara.iyer (write on prod-k8s, 23d after last day)

  before (tracker): 12/12 evidenced, 0 issues, audit gets booked
  after  (gate):    14 findings across 7 controls, pack blocked
12/12 → 5/12controls still clean (tracker → gate)
14findings on a pack with 0 gaps
34 of 53roster covered by the access review
8live grants held by leavers

The clean pack passes untouched — 0 findings, exit 0. A gate that cries wolf on good evidence gets switched off in a week.

② Four ways an attachment count lies
kindnthe question the count never asks
stale4When was it collected? Newest artifact per control aged against the control's own refresh_days window
population1Who does it cover? Reviewed identities diffed against the authoritative roster
artifact_type1What does it prove? required_artifact_type versus what is actually attached
contradiction8Does the rest of the estate agree? Terminated identities joined to the live grant log
stale · VM-01: newest artifact EV-023 collected 2026-03-10 is 148d old against a 30d window (118d past due)
population · AC-03: review covers 34/53 of the roster (64.2%); 19 identities never in scope: contractor x11, service_account x8
artifact_type · BC-01: 2 artifact(s) attached, all of type backup_job_log — the control requires restore_test_report, of which there are 0
contradiction · cara.iyer (engineer) terminated 2026-07-13 — 23d ago — still holds write on prod-k8s, last used 2026-07-05

Every finding names the number behind it, not just the control id — the window, the age, the days past due; covered over total plus who is missing and in what role; the type required against the types present; the person, their last day, the system, the privilege, the days elapsed. A finding that says "evidence is stale" gets argued with; one that says 118 days past a 30-day window does not.

③ Verified, not asserted
8 passedpytest — each sabotage class asserted with its numbers
0 / non-zeroclean pack exits 0, shipped pack blocks
fixtures deterministic — CI regenerates and diffs

The tests also assert the naive attachment count misses all four classes, so the finding the gate prevents is demonstrated rather than claimed. Fixture scale: 12 controls · 24 artifacts · 60 identities (53 active) · 90 grants, generated from a seed by scripts/make_data.py — CI regenerates them and fails if the result isn't byte-identical.

Why compliance evidence

The controls are usually written correctly. The evidence is where readiness quietly fails.

What breaks is the four dimensions no attachment count measures: when the artifact was collected, who it covers, what it actually proves, and whether the rest of the estate contradicts it. Those are data-engineering questions — date windows, population diffs, joins — not legal ones, and they are cheap to answer continuously against exports most organisations already produce from Okta, AWS, GitHub, Jira and their HRIS. Finding a 19-identity population gap in March costs a script run; finding it during fieldwork costs a re-test.

Run it as a gate: soc2_evidence.run exits non-zero on any finding, so it slots into a nightly job or CI and the pack is checked continuously instead of the week before fieldwork. --report out.json writes the findings as structured JSON for a ticket queue.

Reproduce it
make test        # 8 passed
make demo        # the tracker-vs-gate contrast above
make gate        # the shipped pack — exits non-zero, pack blocked
make gate-clean  # the no-false-alarms case — exits 0
python -m soc2_evidence.run --controls data/controls.json --evidence data/evidence.csv --as-of 2026-08-05
Honest limitations
· Synthetic data (12 controls · 24 artifacts · 60 identities · 90 grants) — a demonstrator of the
  method, not a report on any real organisation. No client data appears here.
· This checks whether the evidence is complete. It does not judge whether a control is designed
  adequately — that is a licensed CPA firm's call, and nothing here is an opinion or a certification.
· Control definitions are hand-written JSON. Mapping a real framework, control versioning, and
  exceptions / compensating controls are out of scope.
· It checks metadata and populations, not artifact contents: it cannot tell a genuine restore-test
  report from an empty file named like one. Chain of custody is the next layer, deliberately skipped.
· Four checks are the common failure modes, not a complete taxonomy — sampling adequacy,
  segregation of duties and vendor evidence are not covered.
· It reports findings; it does not remediate. Revoking the orphaned grants is still a human decision.
This is a synthetic sample demonstrating the method. Inspect the four checks, the deterministic fixtures, and the test suite ↗