In the tutorial, the AI call always works. You import the SDK, paste in an API key, await a completion, and render the result. Ship it.
In production, that same line of code is a network call to someone else's infrastructure, running someone else's capacity planning, under someone else's incident response. Sometimes it's slow. Sometimes it rate-limits you. Sometimes it's just down.
The question isn't whether your AI provider will have an outage. It's what your users see when it does. If the answer is "a frozen screen" or "Something went wrong," the problem isn't the provider. It's the architecture.
Here are the failure modes I see most often, and what graceful degradation looks like for each.
Illustrative scenario: a project management app adds an AI-generated summary at the top of every ticket. The summary is fetched server-side before the page renders. One afternoon the provider's latency jumps from 2 seconds to 40. Now every ticket page takes 40 seconds to load, including for users who never read the summary.
Fix: never put a model call on the critical rendering path. Load the core page first, fetch AI output asynchronously, and give it a hard timeout. If it doesn't arrive, the slot stays empty or shows a quiet "summary unavailable." The ticket still opens.
If your code calls one vendor's SDK directly in twenty places, you have twenty single points of failure and no switch to flip.
Put a thin routing layer between your app and the providers, and make fallback a first-class behavior rather than a try/catch afterthought:
Two details matter more than the loop itself. The circuit breaker stops you from hammering a provider that's clearly down (and paying the full timeout on every request). And null is a legitimate return value, which forces every caller to decide what happens without AI.
A small local model (served through something like Ollama or llama.cpp) is a great last line of defense, but it isn't a drop-in replacement for a frontier model. Prompts tuned for one often produce garbage on the other.
Decide in advance which tasks are essential enough to fall back locally: classification, short extraction, simple rewrites. Give those their own prompts and test them. Everything else should degrade to "not available right now" rather than to a confidently worse answer.
This is the one that turns a provider outage into your outage. Search that only works through embeddings. A form that can't be submitted until the AI validator responds. An onboarding flow that stalls because the welcome message is generated.
Every AI feature should have a non-AI path that still lets the user finish their job: keyword search behind semantic search, rule-based validation behind AI validation, a static template behind the generated one.
Graceful degradation is really just treating your AI provider like any other external dependency: timeouts, fallbacks, circuit breakers, and a clear answer for "what if it's gone?" The teams that struggle are the ones who treated the model as part of their own code instead of as a service they rent.
Retrofitting this later means touching every call site. Designing it in from the start means one routing layer and a few decisions.
What does your app actually do today when your AI provider goes down? Have you tested it, or are you assuming?