A FinOps pass over your LLM usage log: reads a month of API calls + a pricing table and reports exactly how much spend is recoverable, with the dollar figure behind every recommendation — not a vibe. Offline, auditable, zero dependencies.
Public · synthetic demoLLM bills creep. The fix is real, but which calls should move, and how much would it actually save? Without dollars attached, cost work stays a good intention. This turns a usage log into a ranked, priced list of what to change.
baseline spend $ 52.16 optimized spend $ 31.86 recoverable $ 20.30 ████████████·················· 38.9% ------------------------------------------------------------ · right-size premium→cheaper : $ 19.63 (8500 calls) · prompt cache (repeat prompts): $ 0.67 (2838 cache hits) ------------------------------------------------------------ biggest single win: 'classify' on gpt-4o = $17.32 (33.2% of spend) → move 6000 calls to gpt-4o-mini: save $16.28
Dollars are small because the log is small — the percentage and the ranked findings port to a real invoice (spend scales linearly with volume).
RIGHT-SIZE simple tasks (classify/tag/…) on a premium model → cheaper sibling.
Legit draft/reason calls are left alone — no saving on real work.
PROMPT CACHE identical prompts repeated: every repeat after the first is a
cache hit. Applied AFTER right-sizing, so the levers compound honestly.
The cache figure is small here precisely because right-sizing already moved those calls to a cheap-input model — shown, not hidden. Point it at your own export + rates and the same math runs.
Every dollar is reconstructable from pricing.json; tests pin the cost math and lever consistency, and CI regenerates the usage log from a seed and fails if it isn't byte-identical.
data/usage.csv model, task, prompt_tokens, completion_tokens, prompt_hash data/pricing.json your account's rates + downgrade map — every figure follows from it
· Synthetic usage log and representative prices — a demonstrator. Point it at your real export for real numbers. · Right-sizing assumes simple tasks tolerate the cheaper model — validate quality with an eval harness before flipping (cheaper is only a win if the output still passes). · Two levers here; a full practice adds batching, max-token trimming, prompt compression, and commitment discounts. · Cache savings assume repeated prompts are genuinely cacheable (deterministic, same context).