Reads your Airflow DAG files with ast — no Airflow install, no imports, no side effects — and reports what a green DAG in the UI will not. The sabotaged DAG here is valid Python that parses without warnings. It also queues 90,816 backfill runs the moment it deploys, writes revenue twice from two unordered tasks, and pages nobody when it fails at 3am.
Nothing about broken_pipeline.py looks wrong in review. It parses, it renders, the graph makes sense. The defects are all configuration — a missing max_active_runs, an append-only write with retries enabled, two writers to one table with no edge between them. They only become visible in production, usually at the worst moment.
planted defects : 12 caught : 12 missed : none false alarms on clean: 0 OK 12/12 caught, 0 false alarms
Exit code 1 on high or critical, so CI blocks the merge. The clean DAG exits 0 and the same job stays quiet.
O1 computes the actual number. catchup=True is not a defect by itself — it is a defect when start_date is far in the past and the schedule is short. The check does the arithmetic: (now − start_date) / interval. Here that is 90,816 runs. A reviewer skimming the file sees catchup=True and moves on; the number is what makes it real.
O6 and O11 are a pair. O6 says the write is not idempotent. O11 says two of them run with no ordering. Either alone is survivable. Together they mean the revenue number depends on scheduler timing — the kind of bug that gets found in a board meeting.
O10 is about worker economics. A sensor in poke mode holds its slot while it waits. With a 15-minute schedule and a 24-hour timeout, every run parks a worker for up to a day. The pool fills with sensors waiting for upstreams that are themselves waiting for a free slot.
| # | severity | what it catches |
|---|---|---|
| O1 | critical | catchup=True over a long backlog — 90,816 runs queue on deploy |
| O3 | critical | dependency cycle — the DAG can never complete |
| O5 | critical | no max_active_runs on a short schedule — two writers, one table |
| O6 | critical | append-only write a rerun duplicates — silently double-counted revenue |
| O11 | critical | two writers to one target with no edge — row count depends on finish order |
| O2 | high | retries not set — one 503 ends the whole run |
| O7 | high | depends_on_past with no retry budget — one failure blocks every future run |
| O9 | high | no execution_timeout — a hung request holds a worker slot forever |
| O10 | high | sensor waits longer than the schedule interval in poke mode |
| O12 | high | no on_failure_callback — it fails at 3am, someone notices at 9 |
| O4 | medium | task with no dependencies either way — readers may get the stale version |
| O8 | medium | no SLA anywhere — late is not failed, so a stale dashboard looks live |
Every finding carries three fields, not one: evidence (what is in the file), impact (what happens in production), fix (what to change). A finding that only says "missing retries" gets ignored; one that says what breaks does not.
Importing executes module-level code, so it needs the whole Airflow install, every connection the file touches, and any credential a top-level client happens to construct — a slow, fragile CI job that fails for reasons unrelated to the DAG. dag-guard parses with ast instead: it runs on a plain python:3.11-slim image in about a second and cannot have side effects.
The tradeoff is stated honestly: it sees what is written in the file, not what a dynamically generated DAG resolves to at runtime.
python3 tests/test_checks.py # the 12/12 assertion above ./scripts/run_evidence.sh # full output for both DAGs into evidence/ python3 -m dagguard.cli dags/*.py --now 2026-08-04
· Static analysis: dynamically generated DAGs resolve at runtime and are out of reach. · Synthetic DAG pair (one clean, one with 12 planted defects) — a demonstrator, not a benchmark. · It checks the orchestration. Whether the data itself is correct is warehouse-quality-gate; whether a job silently moved zero rows is pipeline-heartbeat.