I build data and AI-automation pipelines with a verification layer built in — confidence scores, checks against source, and a human-review queue for anything uncertain. Then I show what the checks caught, what they missed, and where a person still has to decide. Anyone can get a model to emit JSON; what a team pays for is output it can trust and sign off on.
Load a CSV or JSON in the browser and get contract violations and drift signals against an editable, inferred contract. Nothing is uploaded — every check runs on the page.
Compare a source and a target export and get key reconciliation plus field-level evidence — the step where a migration quietly breaks. Both files stay in the browser.
A dark, interactive WNBA player-prop board — sortable table with per-player sparklines, book line vs. projection with over/under coloring, and a click-through player-detail panel. Deployed as a standalone Vercel page that auto-updates on every push.
Before you pay for a dataset, get a verdict on it — episode completeness, sensor synchronisation, joint and gripper ranges, duplicate rate. You get a stated pass line, the reason behind every failed episode, and a command that reproduces the whole audit.
LeRobot v2.x → v3, and conversions between RLDS, HDF5 and LeRobot layouts. The migration is the step where datasets quietly break, so this one ships with field-by-field evidence that it held — not just a script that exited zero.
Can these datasets safely be trained together? Did this policy actually beat that one? Per-pair normalisation plans from the manifests, and success rates with Wilson confidence intervals instead of a single number.
A (synthetic) ROS2 rosbag export of robot telemetry → a structured anomaly audit: per-topic publish-rate deviations, dropout gaps, header-vs-receive clock skew, out-of-range sensor values, TF frame gaps, and a cmd_vel-vs-odom stall signal — organized by how sure the evidence is.
A ROS2 colcon workspace → a reproducible Docker build + GitHub Actions CI (colcon build/test) → an offline Python layer that parses the build/test logs into an evidence-graded build-health report.
A robot URDF → static structural audit (link/joint tree integrity, missing meshes, inertial sanity, joint limits) → a four-section smoke-test report.
Raw operator/rosbag logs → a training-ready ML dataset (RLDS/LeRobot-style) with episode-level splits, a data contract that fails the build on bad data, and an RLDS-vs-LeRobot format recommendation.
A LeRobot v2.1 dataset → a validated migration to the v3 layout: documented field/layout mapping, then integrity checks (frame counts, features, index continuity, stats) so nothing is silently dropped.
A robot/ML dataset manifest → near-duplicate detection (Hamming/cosine on provided hashes), quality gates, a coverage/imbalance report, and a curated manifest with a justified rejection log.
Robot policy rollout logs → success rate with Wilson confidence intervals, per-task breakdowns, and A/B comparison (difference CI + two-proportion test), with small-sample flags.
Multiple robot demonstration datasets (different embodiments, action spaces, control rates, gripper conventions) → a mixability assessment: normalize, flag frequency/gripper conflicts and missing embodiment metadata, and a per-pair mix / mix-subset / do-not-mix report.
A parameter-space spec → a domain-randomization sweep (grid / random / Latin-hypercube) plus coverage and pairwise-gap analysis, ready to feed a simulator.
Raw lead / form submissions → normalized, classified, and scored (hot / warm / cold) → a staff summary and reply draft → routed by a verification gate: trustworthy leads auto-flow to the CRM, uncertain ones go to a human-review queue. Runs fully offline (deterministic) or with OpenAI.
An n8n workflow that scores incoming inquiries with deterministic rules and routes them through a verification gate: complete leads land in a CRM-ready lane, uncertain ones stop in a human-review queue with explicit reasons, spam is rejected. Ships with the workflow JSON, a 24-case golden fixture set, ten unit tests and run screenshots — and the same scoring logic runs standalone with zero dependencies.
Leads arrive from four differently-shaped sources (web form, Facebook Lead Ads, inbound email, CSV) → normalized to one schema → cross-source identity resolution merges the same person across channels → deterministic routing / SLA → a review gate for anything uncertain. Runs fully offline (deterministic) or with OpenAI.
A permission layer in front of MCP tools, and a log that can prove what it decided. The same 17-call session through a naive allowlist and through nine checks: calls executed without a grant 7 → 0, decisions that cannot be reconstructed from the log 16 → 0, denials that left no trace 1 → 0, secrets sitting in the log 2 → 0, entries removable unnoticed 1 → 0 — with 0 legitimate calls refused.
--verify re-decides all 17 entries from the log alone and checks each against the verdict it claims. Redaction at write time, hash-chained entries, and a stated limit: this is a policy and audit layer, not a sandbox.An LLM support agent that cannot spend money without a human — and the number that proves it. The same 37 recorded incidents run twice, once through a typical agent loop and once through seven gates: unapproved money moved $453.50 → $0.00, duplicate side effects 4 → 0, injected args executed 4 → 0, malformed money moves 2 → 0, and 5 actions held for a person.
policy.yaml to ask whether a human was required. Noise on control tasks stays at 0 — legitimate work is untouched.An MCP server where every read names its evidence and the one write is impossible without an approval token. A risk envelope checked before a human ever sees the request, a single-use token issued by a separate step, a publish that cannot happen without it, and an append-only log of every attempt — especially the refused ones.
odds-consensus, so the integration surface is a real pipeline rather than a mock.A narrow Python guard for audit, inventory, and verification tools that are meant to read evidence without changing the environment they are pointed at. Blocks write-capable open modes, selected pathlib/os/shutil mutations, and HTTP POST/PUT/PATCH/DELETE — while the sample inventory still runs inside the guard.
Static-scans a codebase for retired and soon-to-retire LLM model IDs, API surfaces, and parameters, then reports days remaining, blast radius, and the published replacement path. The deterministic demo catches 11 of 12 planted cases; the twelfth is a current-model negative control.
Scans a repo into a CLAUDE.md / AGENTS.md an agent can follow — build/test commands, coding rules, prohibitions, CI smoke test, Definition of Done — then detects when it goes stale and scores readiness deterministically.
Golden-set regression testing for LLM / extraction pipelines: scores a pipeline, judges the fields where exact match is unfair, and fails CI when quality regresses. A real amount-parsing bug drops row-correct 100% → 48% and the gate blocks it.
A hardened MCP server exposing read-only order tools to an AI agent — treating every tool argument as untrusted, because it's LLM-generated. Validated, parameterized, read-only. A naive concat server leaks the whole table on a classic OR-1=1 injection; this one blocks 5/5 with zero functionality lost.
A FinOps pass over an LLM usage log: reads a month of calls + a pricing table and reports how much spend is recoverable, with the dollars behind every recommendation. Model right-sizing + prompt-cache savings, runnable as a spend gate.
A drop-in input/output guardrail layer for LLM apps: blocks prompt injection and jailbreaks before they reach the model, and redacts leaked secrets/PII before they reach the user. Naive lets 33/33 attacks+leaks through; guarded stops all 33 — with zero false positives on 17 benign lookalikes.
The reliability layer an agent’s tool calls need — retry, timeout, circuit breaker, idempotency — with the numbers showing what each buys. Same workload through naive / retry-only / reliable: naive 53/80 → reliable 63/80, double-charges 5 → 0, load on a down service 88 → 30 calls.
A document copilot that cites what it says, refuses when the evidence is not there, and ships the test that proves both. Same corpus, same 33 questions, same scorer: baseline 12/33 → harness 32/33, including 4/4 on superseded editions, 4/4 on regional scope, and 4/4 on questions the corpus cannot answer at all.
network_mode: none.A support bot’s mistakes do not live in its sentences — they live between them. The layer between a model’s draft reply and the person reading it, scored over 17 conversations / 32 turns: conversations never handed off 4 → 0, replies that must not go out but were sent anyway 4 → 0, good conversations interrupted 3 → 0.
B-06 alone and it is a correct, well-cited sentence; read it after turn 1 and it is the bot telling the same person the opposite of what it just said. The naive baseline — a keyword escalation list — escalates the wrong half and misses every cross-turn failure.Documents (invoices, purchase orders, bank statements — PDF or scanned) → structured JSON/CSV through a deterministic verification engine: line math, balance reconciliation, totals, required fields, confidence scores, and a cross-document human-review gate. Optional AWS Textract path for scans.
Screenshots → a spreadsheet, with a verification layer between them: per-field confidence gates, type and pattern checks, and a subtotal + tax = total reconciliation that catches OCR errors which parse cleanly. Ships with a Google Apps Script + Cloud Vision deployment path, so the client never touches the code.
Grounded document Q&A that cites its sources, refuses when unsure (no hallucinations), and ships with an evaluation harness — recall@k, refusal accuracy, and grounding rate.
Cross-document Q&A that surfaces every source's claim and flags contradictions — old vs new policy, US vs EU handbook — routing conflicts to human review instead of hiding them.
Hybrid retrieval — BM25 + semantic, fused with Reciprocal Rank Fusion and reranked — with an evaluation harness that measures the lift (recall@k, MRR, nDCG) over any single retriever.
De-margined fair prices and a cross-book consensus that refuses to merge markets that are not the same. One schema for five books, the margin stripped the same way everywhere, and a hard rule for when two markets are actually the same market — before anyone claims a book is off the market by 9.85%.
skipped.json and devig_skips.json record what was refused and why, so the consensus is a statement about markets that were genuinely comparable. No keys, no network, byte-identical fixtures.Messy CSV → cleaned data (with a quality log) → KPIs and month-over-month analysis → an automated report with charts. Includes anomaly detection: outliers, revenue mismatches, MoM spikes.
Customer-churn prediction with an honest evaluation: leakage-safe splits, a baseline to beat, 5-fold cross-validation, and a model card that states the limits. Logistic regression implemented in NumPy.
Market-basket association rules (Apriori, lift-based) + RFM customer segmentation — surfacing real cross-sell signals and at-risk customers, with explainable segments.
Site photo → object detection → checklist verification → a PASS / REVIEW / FAIL report with annotated images. A confidence gate routes uncertain calls to human review; the detector is swappable (offline or a production vision model).
Bank and card statements into a reconciled ledger — and a refusal, in plain English, for the ones that do not add up. Importers are scored on rows parsed; this is scored on rows wrong: silently wrong rows 26 → 0, silently wrong money $62,832.40 → $0.00, unreconciled $8,905.20 → $0.00, corrupt sources caught 0/5 → 5/5.
Decide what your duplicates are before you delete any of them. The three-line DELETE … WHERE id NOT IN (SELECT MIN(id) …) against a gated path over eleven cases: facts lost 7 → 0, rows destroyed 6 → 0, rows fabricated 4 → 0, duplicates left behind 4 → 0, deletes with no undo 13 → 0, blocking operations in the plan 22 → 0.
external_id are not one person, and the gate says so instead of picking a winner — 3 held for a human, 1 migration refused, and 0 false alarms on the sets that really were duplicates.A run-tracking table that has never once said “failed” is not evidence that nothing failed. 282 runs across four pipelines, every row marked success or still running, reconstructed from what the runs left behind: silent failures found 4/10 → 10/10, false accusations 4 → 0, duplicate alerts 3 → 0, runs that never ran 0/2 → 2/2.
Tell me what changed on this page — not what moved on it. Change watchers fail in both directions at once, and both are quiet: over the same 12 material changes, missed changes 7 → 0 and false alarms 40 → 0, with unusable snapshots caught 0/5 → 5/5.
robots directive.Scheduled, offline evidence for a data operation that may drift after delivery. Replays four committed weeks of an order feed through six stages — collect, normalize, declared checks, change record, alert payloads written but not sent, and exports a client can open without this codebase.
Two synthetic before/after repair cases with the fault planted on purpose: a pipeline that completes without an exception while mapping reordered columns by position and misreading a changed encoding, and a static page carrying a blocking script, four font stylesheets, and an injected layout shift.
Messy product listings → clean, validated, de-duplicated data. LLM + heuristic extraction with a per-row confidence score and human-review flags.
Multi-source scraper + unified-schema normalizer for robotics/electronics parts (Pololu + Adafruit, 380 products), with idempotent ETL into SQLite.
Seven invariants that fire when a site changes underneath a working scraper — selector hit-rate, per-field fill-rate, a median+MAD row-count band, type and enum stability, tag-skeleton drift, and freshness. Runs offline against committed snapshots of a fictional catalogue.
An offline run-ledger monitor for scheduled pipelines — liveness, missed schedules, empty success, median+MAD volume and runtime bands, schema and watermark drift, partial sources, duplicate batches, persistent finding state, and a period digest.
Reproducible Python pipeline building a WNBA player-vs-defense info sheet — player game logs joined to opponent defense, every field verifiable against source.
An audit of a no-code automation’s own execution history — for the runs that reported success and still lost data. Silent attrition at a filter step, duplicate delivery from a retry without idempotency, schedule gaps, retry storms whose eventual success hides the failures, success rows carrying an error payload, and events applied out of their own order.
A three-way reconciliation for an online store — orders against fulfillments against payouts. Paid but never shipped, shipped but never paid, quantity mismatches, payout shortfalls attributed to fee drift or an unrecorded refund, duplicate charges separated from legitimate repeat purchases, refunds with no restock, and currency or cent-level rounding breaks.
Decimal end to end, with a test asserting no float ever touches it. A clean set with a real repeat purchase and a real partial fulfillment passes at 0.A load contract for the spreadsheet that quietly feeds a production system — header drift including reordering, spreadsheet error literals arriving as text, type drift, ambiguous dates, leading-zero and scientific-notation mangling, structural noise like repeated headers and totals rows, primary-key integrity, and declared enums and ranges.
A pre-flight gate for a legacy export on its way into Odoo — required fields, dangling many2one references, duplicate and near-duplicate external IDs, selection values outside the allowed set, non-ISO dates, thousands separators in decimals, credit-note sign convention, and header totals that disagree with their own lines.
amount_total and the sum of its lines, an EUR invoice with no rate against a USD company, two references pointing at partners that are not in the export, and ‘15/01/2026’ parsed as a date. A clean export passes at 0.A load contract for GeoJSON before it reaches PostGIS — CRS declaration, coordinate range, ring closure, winding order, self-intersection, zero-area and too-short rings, hole containment, duplicate geometry, dimension and geometry-type conformance, plus a longitude/latitude axis-swap heuristic.
Deduplication and merge audit for contacts a CRM already holds — blocking, per-field similarity, a three-way verdict, declared conflict-resolution rules, and an audit log that names which record won every field and why. Nothing is looked up externally; the tool reads one file and writes cleaned output.
A post-migration verification gate for the losses a row count cannot see — primary-key reconciliation plus column-aggregate parity between the legacy table and the new warehouse. Row churn that nets to zero, a lost numeric scale on the money column, a timezone shift onto the wrong calendar day, silent truncation and NULL→'' coercion.
An evidence completeness gate for SOC 2 / ISO 27001 control sets — refresh windows, population coverage against the authoritative roster, artifact-type matching, and contradiction against the access log. Evidence engineering: it checks whether the pack is complete, it does not issue an audit opinion.
A pre-release gate for “anonymized” exports — column-agnostic PII detection, mask-uniqueness and hash-invertibility checks, k-anonymity over quasi-identifiers, and DSAR erasure completeness across every related table.
A schema + range + enum contract with drift detection for tabular & sensor (rosbag) batches. Catches the batch that still parses but whose meaning changed — a firmware unit switch (°C→°F), a new category, a sensor dropout — that a naive "did it parse?" gate ships.
A dbt contract — 15 tests over staging and marts — that blocks a bad batch before it reaches the mart. Duplicate keys, broken relationships, a cents/dollars unit mix-up, a mixed-currency batch, an out-of-window date, and a status with the wrong case.
Static review for Airflow DAGs — 12 checks read the source with ast, so there is no Airflow install, no import, and no side effects. Catchup blast radius, dependency cycles, non-idempotent writes, unordered co-writers, missing timeouts, retries, SLAs and failure callbacks.
A single YAML contract for six business metrics, and a checker that diffs every dashboard against it — filter sets, timezone, currency handling, distinct-ness, expressions, rate denominators, unowned metrics and dead ones.
Twelve semantic checks for FHIR R4 bundles that are already structurally valid — code-system mislabelling, ICD-10 without its decimal, UCUM units that contradict the LOINC code, implausible values, unresolvable references, naive timestamps and value-set casing.
A release gate for automatically-marked answer sheets. It grades against the rubric, then decides which of those grades are supported well enough to reach a student — naming the criterion, the reason, and the words on the page for every one it holds.
A raw event/transaction log → scheduled daily/weekly reports with period-over-period deltas, threshold + statistical-outlier alerts, and idempotent re-runs.
A demand time series → forecasts with prediction intervals from a rolling-origin backtest, honest error metrics, and an explicit "when NOT to trust this" section.
A transaction log → sequential purchase patterns, cohort retention, and churn associations, with Wilson intervals so small groups aren't over-read.
Experiment/metric data → honest significance testing: two-proportion z-test and Welch's t-test (stdlib), confidence intervals, multiple-comparison correction (Bonferroni + Benjamini-Hochberg), and minimum detectable effect.
Tagged site-photo metadata (EXIF) + an inspection checklist → a construction progress report: percent complete per area/phase, a timeline, and missing / out-of-sequence / stale-area detection.
A plan/spec with tolerances vs a measured inspection log → a reconciliation report: matched items, missing, extra, and out-of-tolerance deviations, each with the numbers.
MLB game logs → a reproducible batter-vs-pitcher info sheet: season/L5/L10, home-away and vs-hand splits, and head-to-head history, every number traceable to its sample size.
A single-agent loop — planner → guardrail validation → executor → eval harness. Tool registry with arg schemas, allow/deny + approval gates for side-effecting tools, step caps, and loop detection.
A supervisor decomposes a task and routes subtasks to specialized worker agents over a message bus + shared blackboard, with coordination guardrails.
Wraps an agent run to produce a structured trace (steps, tokens, latency), cost accounting from a pricing table, hard token/$ budget guards, and failure policies (retry, circuit-breaker, fallback).
Agent execution traces → a rule-based failure taxonomy (tool-selection error, loop, timeout, malformed args, wrong answer…), a failure distribution, and a labeled eval-ready dataset with a human-review queue for ambiguous cases.