“I Barely Use It” — Where $100 of AI API Credit Actually Went

I run a self-hosted AI assistant for a small business. It reads a shared mailbox, files documents, answers questions in chat and runs a few daily research jobs. One morning every call started failing with insufficient_quota. The prepaid $100 API credit was gone.

The owner's first reaction: "I barely use it." He was right. Almost none of that money went to him actually talking to the thing.

TL;DR: Nearly all the spend came from machine-to-machine work: a heartbeat firing every 30 minutes around the clock, scheduled jobs running on the premium model, and one retry storm that hammered the API for hours. The interactive chats were a rounding error. Fix: cheap model for everything in the background, premium only where a human waits for an answer, and hard caps on anything that can loop.

Gotcha: an empty quota doesn't send you an email

When the credit ran out, nothing crashed loudly. Every LLM call just failed with a quota error, the agent's fallback model failed too (same account), and the user saw a bot that "stopped answering." Failed calls cost nothing, so the bill stopped growing, and so did the usefulness.

So the first job wasn't saving money. It was finding out what had spent it.

Find the burn before you touch the config

The provider dashboard tells you how much. It doesn't tell you which part of your setup. For that I wrote a small read-only script that parses the agent container's logs and breaks every model call down by:

• model: which calls hit the expensive one
• hour of day: a flat line at 3am means a loop, not a person
• lane/purpose: interactive chat vs. scheduled job vs. background
• repeats: the same input processed over and over

# illustrative — adapt container name + pattern to your log format
# counts model calls per hour and model
docker logs --since 168h agent-gateway 2>&1 \
  | grep -oE '^[0-9-]+T[0-9]{2}.*model=[a-z0-9.-]+' \
  | sed -E 's/^([0-9-]+T[0-9]{2}).*model=(.*)/\1 \2/' \
  | sort | uniq -c

Seven days of that made the picture obvious. There were four drivers, and none of them was "the owner chats too much."

Driver 1: a heartbeat nobody needed

The agent framework has a heartbeat. Every 30 minutes the agent wakes up, reads a task file and checks whether there's anything to do. That's 48 runs a day, 24/7, on the premium model.

The task file contained… comments. No actual tasks. So 48 times a day, a top-tier model was paid to read an empty to-do list and conclude there was nothing to do.

This is the classic "runs even though nobody uses it" cost. It doesn't show up in anyone's mental model because nobody triggered it.

Fix: heartbeat to once every 24 hours, on the small model. Checking an idle list doesn't need the expensive model.

Driver 2: scheduled jobs on the premium model

There were five daily scheduled jobs: document scanning, a few research tasks, a daily question prompt. Each one is a multi-step agent run. It searches, reads pages, calls tools and summarizes. Every step is a billed model call, and all of them ran on the premium default because that was the global setting.

Fix: pin every scheduled job to the small model explicitly. Most agent frameworks let you override the model per job; use it. A daily research digest is rarely worth premium pricing.

Driver 3: classification on the premium model

Every incoming email got classified ("is this an action item or just CC?") by the premium model. Classification is exactly the kind of narrow, repetitive task small models handle fine.

Fix: classification and fact extraction on the small model. The step that actually does the work (reads attachments, writes the reply) stays on the premium model, because that's where quality is visible.

The good news here: the mail pipeline only costs money per new mail. No mail, no LLM calls, just a cheap mailbox poll. As long as it processes each mail exactly once, that is.

Driver 4: the retry storm (the big one)

On the worst day the logs showed 781 model calls. 302 of them were for one single email.

A downstream step kept failing, the "done" marker was only written after that step succeeded, so the same mail was picked up again every two minutes for hours. Each pass meant classification, a processing run and another failure. I wrote up that bug pattern separately in The Idempotency Bug That Uploaded the Same File 153 Times. For this post the relevant part is what it does to an API bill: a loop in an AI pipeline multiplies cost by however long it takes you to notice.

Fix, in two layers:

1. Idempotency: mark the item as processed right after the irreversible step, not after the fragile one.
2. Circuit breaker: a per-item attempt counter, max 3. Even if a future bug breaks idempotency again, one item can't trigger more than three runs.

MAX_ATTEMPTS = 3

attempts = state.setdefault("attempts", {})
n = attempts.get(msg_id, 0)
if n >= MAX_ATTEMPTS:
    log(f"skip {msg_id}: {n} attempts, needs a human")
    continue
attempts[msg_id] = n + 1
save_state(state)          # persist BEFORE the expensive call
run_agent(msg_id)

After the idempotency fix, daily volume dropped back to a baseline of roughly 80 calls a day, about a tenth of the storm day.

Gotcha: "just use the small model everywhere" backfired

Once the drivers were clear, the tempting move was to flip the global default to the small model and be done. I tried exactly that.

Costs went down. Quality went down in ways that took a few days to notice: the assistant stopped writing things into its long-term notes, and its daily questions started repeating. Nothing errored. It just got quietly worse at the parts that need judgment.

So I reverted the interactive default to the premium model and kept the small model pinned on everything in the background. That split is the actual lesson:

What I'd set up from day one next time

• Per-job model pins. Never let background work inherit the premium default.
• A cost breakdown by purpose, not just a total. If you can't answer "how much did the heartbeat cost this week," you're guessing.
• Circuit breakers on every loop that calls an LLM. Max attempts per item, persisted before the call.
• A budget alert at the provider. I only learned the credit was empty because the bot went silent. A low-balance alert would have flagged the storm on day one.
• Check the heartbeat's task list. If it's empty, the heartbeat is pure cost.

None of this is exotic. The trap is psychological: with AI assistants, people think in conversations, but the bill counts calls, and most calls are made by your own automation while everyone's asleep.

Related posts

• My AI Receipt Sorter Shifted Amounts by 2,000 Euros, So I Took the Numbers Away From It
• My AI Agent Invented a CLI Flag, and I Called the Error "Cosmetic"
• My AI Agent Only Saw 71% of Its Own Memory