Reliability · 2026-11-12 · 321AI engineering notes · get a key
Quick answer: Multi-source failover is the difference between an upstream hiccup and your outage. On a multi-supplier marketplace, a failing lane is replaced mid-flight by the next healthy lane quoting the same model — automatically. Here is exactly what happens, what it does not promise, and how to verify it yourself.
A single supply lane fails in familiar ways: 429 throttling under load, 5xx errors from upstream incidents, latency spikes that make interactive apps feel broken, providers delisting or re-pricing a model, and the occasional operator-side outage. None of these are exotic — they are a normal week. The question is not whether a lane fails but whether its failure has to become yours.
The first two — throttling and 5xx — are transient, and exactly what automatic failover absorbs. The last two are structural: a delisted model does not come back on the same lane, and capacity contracted away does not either. Supplier depth per model is the hedge for all four, which is why the catalog treats supplier count as a first-class column rather than a footnote.
Because several suppliers quote the same model, "the next healthy lane" usually exists — that is what supplier depth buys. The catalog’s supplier-count column tells you how deep that reserve is per model before you ever need it.
None of this requires your participation. There is no webhook to subscribe to, no health-check endpoint to configure, no fallback key to wire — the mechanism runs on the platform side, which is the point: reliability that depends on every customer implementing it correctly is not reliability.
| Failure | Single-lane outcome | Multi-source outcome |
|---|---|---|
| 429 throttling | Your errors, your retry logic | Rerouted to the next healthy lane |
| Upstream 5xx incident | Outage until fixed | Traffic moves; failed calls refunded |
| Latency spike | Slow product | Scoring prefers faster healthy lanes |
| Model delisted | Integration breaks | Same model from other lanes; catalog updates |
Honesty matters here, and the Terms of Service are explicit: multi-supplier failover is best-effort, and no platform can guarantee the permanent availability of any single model — suppliers may throttle, delist or change models. For outages caused by force majeure, upstream failures or maintenance, the platform restores service as fast as it can and handles impact under the unconsumed-balance-is-refundable principle. Failover dramatically shrinks the blast radius of a lane failure; it does not repeal physics.
Three public sources: the status page for the live platform snapshot; the supplier count and average stability columns on the catalog for per-model resilience; and the free stats API for programmatic checks. Your own monitoring should track first-token latency and error rate per model — after a failover-worthy event, compare notes. The trust page documents the self-healing mechanics this article summarizes.
Failover shifts work from your pager to the platform, but a few client habits still pay: set request timeouts generous enough for one reroute (not so generous that a dead lane eats your SLA), use retry-safe request design so a retried call cannot double-execute, and alert on error rate per model rather than per HTTP status. If your SDK exposes middleware, log the lane metadata that comes back with responses — after an incident, that log answers "did failover actually happen?" in seconds instead of an argument.
Failover is automatic, but its preference is yours: the stable token group prioritizes measured stability for user-facing keys, while default favors price for batch work. Critical flows that cannot tolerate model mismatch run on guaranteed — merchant-supplied lanes. See the FAQ for the group semantics and the guides for client setup.
Whichever group a key uses, the mechanics underneath are identical — detect, retry, reroute, quarantine, re-probe — only the preference ordering changes.
No. The request is retried on another lane serving the same model; your client sees a normal completion. Failed attempts are never billed.
Persistently bad lanes enter a 1-hour cooldown after repeated failures, are re-probed every 3 minutes, and only return to rotation after 2 consecutive successful probes.
No platform can guarantee the permanent availability of any single model — suppliers may throttle, delist or change them. Failover is best-effort multi-source resilience, and impact during force-majeure events is handled under the refundable-balance principle.
More is more resilient. Check the supplier-count column on the catalog; models quoted by many lanes survive lane-specific failures without any action on your side.
Live prices: /en/models/ · billing: /en/pricing · setup: /en/guides/ · FAQ