Cost optimization · 2026-11-12 · 321AI engineering notes · get a key
Quick answer: Most LLM bills leak in the same five places: paying above the market floor, re-sending unchanged context, calling a big model for small jobs, verbose output, and silent retry waste. Close those leaks and a 50-70% reduction without downgrading models is a realistic target — here is each lever, with illustrative saving bands.
The single biggest lever is what you pay per token. On the marketplace, every model is quoted by several independent suppliers and routing prefers the cheapest healthy lane — the floor is public, so is the count of lanes bidding. Combined with a platform fee of 0, your per-token cost is the market’s current answer, not a vendor’s annual decision. Check the floor column for your stack on the live catalog before anything else.
System prompts, tool schemas, and pinned documents are billed as input on every call unless they are cache reads — and cache reads bill below fresh input. Restructure prompts so the stable content comes first and the volatile part comes last; the stable prefix becomes a cache hit on every subsequent call. For agent loops that re-send instructions each turn, this alone can be the difference between a painful and a boring bill.
Teams default to one model for everything. The bill cares about the fit: hard reasoning earns a flagship, high-volume classification does not. Conditional routes on 321AI re-pick the best lane within 5 minutes as the market moves, and the comparison pages price any two models head-to-head on live data — moving one batch flow to a better-priced tier is a one-line change.
Output tokens bill above input, and generation is where volume concentrates. Ask for structured output, cap max_tokens where full prose is not needed, and make "concise" an explicit instruction with a format. On reasoning models, remember thinking tokens bill as output — a step budget on agentic flows is a budget line, not a style choice.
Audit the usage logs monthly: itemized tokens, model, cost and status per call. Look for retry storms (a misconfigured client hammering a failing flow), abandoned long-context calls, and keys that should not exist anymore. On 321AI failed calls are never billed — but the wasted latency and noise are still yours to find, and the logs make them visible.
| Lever | Effort | Illustrative saving | Where on 321AI |
|---|---|---|---|
| Buy at market floor | None (default) | Depends on your old baseline | Marketplace floor + $0 platform fee |
| Cache stable prefixes | Low (prompt reorder) | Large on repeated-context apps | Cache-read rates in the price API |
| Route by task | Low (model param) | Varies by stack mix | Conditional routes + comparisons |
| Trim output | Low (prompting) | Grows with output share | max_tokens + structured formats |
| Audit usage logs | Medium (monthly) | Finds anomalies, not pennies | Console usage logs |
The bands above are illustrative — your baseline decides the real number. The honest procedure: measure one week, apply one lever at a time, measure again. The calculator re-prices a workload in seconds and the pricing page explains the billing semantics the levers exploit.
Sequencing matters because the levers multiply: floor pricing changes the denominator for everything else, caching changes it for repeated-context flows, routing changes which model the denominator applies to, and output trimming shrinks the expensive term. Applied in that order, each lever's measured saving is visible in the next week's logs instead of blending into one unattributable blob. For client-side wiring of any of this, the setup guides cover the common tools.
It depends entirely on your baseline: teams buying at list price with no caching see the top of that band; teams already on competitive pricing see less. Measure one week, apply levers one at a time, and let your own logs decide.
No — cache reads return the same context at a lower price. The only requirement is structural: stable content first in the prompt, volatile content last.
Lanes serve the same model; upstream self-check records per lane are public so you can see what you are trading. For authenticity-critical workloads, the guaranteed token group pins traffic to merchant-supplied lanes.
Check the floor for your current stack on the live catalog — it takes a minute and requires zero code changes. Then restructure your most-repeated prompt for caching.
Live prices: /en/models/ · billing: /en/pricing · setup: /en/guides/ · FAQ