Start with the work that would otherwise wait.
Imagine a project team preparing its weekly status draft from approved records. A colleague checks the draft before it is used. The ordinary model service is unavailable, so the team considers adding another provider. The useful question is whether that route would let the team finish this task, within its agreed time and quality requirements.
We use this invented example to separate three different problems: the model cannot answer, the application cannot reach either model, or the source records do not contain an approved fact. Each calls for a different response. Buying another model connection addresses none of them automatically.[S1]
First agree how long the draft may wait and who can prepare it manually. A recoverable queue may be enough for a weekly draft; a time-sensitive service could need another design. If the purpose of a second provider is better capability instead, write the specific task requirement it must meet. Our model-selection guide covers that comparison.
Trace both routes through the services they share.
In the graphic, the team keeps its approved records in one source store. Its application sends requests through a shared gateway, an intermediary that directs requests to model A or model B. If only A is unavailable, B could be a candidate. If the gateway is unavailable, this application cannot reach either model through it. Adding a third model behind that same entry would leave this particular dependency in place.
Microsoft’s architecture guidance distinguishes model deployments, hosting instances and regions. It also describes how a gateway can itself become a single point of failure. Stateful interactions may need to stay attached to a particular back end.[S2] In our example, therefore, we would check whether the replacement can obtain the permitted source packet and current work state. A provider-specific conversation or file reference is not a portable copy of its contents.
Draw the actual route rather than counting logos. Include the application, gateway, network, identity service, source store and model host. Mark which components are shared. An alternative path may remove one dependency while retaining others; describe exactly which failure it is intended to handle.
Record the configured processing destinations too. AWS distinguishes geographic from global cross-Region inference profiles, each defining where requests may be routed.[S3] The region in an endpoint name is therefore not a complete description of processing locations. The responsible team must check whether the actual alternate configuration is permitted for the data before testing it.
Compare an extra route with the fallback you already have.
For the status draft, we would compare these choices against the same agreed waiting time, source packet and acceptance criteria. The table specifies questions to resolve; it reports no successful tests or supplier advantages.
| Operating choice | What the team would need to establish | What remains to maintain |
|---|---|---|
| One model service, with a queue or manual drafting path | Can the team recover the request and finish the draft within the agreed time without losing the source version? | Queue visibility, a named person and a workable manual procedure. |
| Another eligible model through the same gateway | Does it meet the task requirements when the first model is unavailable? Which shared failures still stop both? | Both configurations, routing behavior, access and acceptance checks. |
| An eligible service through a separately reachable route | Does that route remain reachable under the specific failure being considered, with permitted data and sufficient work state? | Both integrations, credentials, source access, operational ownership and repeated rehearsals. |
A separate route can be justified, but its upkeep belongs in the comparison. Include integration changes, evaluation, monitoring and the people who handle failed switches. Do not compare only token prices. Anthropic’s advice to start with the simplest sufficient solution supports asking what the extra complexity is for; it does not establish that one provider is always enough.[S4]
Rehearse one failure with a result you can inspect.
Before relying on a fallback, use a permitted nonproduction copy of the task. Record the source version, model configurations, access rules, expected result and named reviewer. Then simulate a specific interruption in that test environment. Keep ordinary work in the test as well, so a path that sends everything to a person does not appear successful merely because it avoids model errors.
For our draft we propose the checks below. The expected behavior is a test specification, not an observed result. The person approving the design should be able to inspect the request, actual response and final disposition together.[S1]
| Test condition | Evidence to retain |
|---|---|
| Model A cannot respond; the shared gateway is available. | Whether B was eligible and reached, which source version it received, its draft, review decision and actual delay. |
| The shared gateway cannot receive the request. | The recorded interruption and the queue or human path actually used. Do not label B an available fallback through that same gateway. |
| The replacement cannot access the required source or work state. | The missing dependency and resulting pause or handover. A fluent draft reconstructed from guesses fails the check. |
| The source contains two unresolved approved dates. | Both dates remain visible and the owner is asked to resolve them. Switching models must not manufacture the missing decision. |
Record charges, delay and human work after the attempt, including failed requests. Check how the queued job is recovered so it is not unknowingly processed twice. If the workflow can also write to business systems, a timeout may leave an action’s outcome uncertain; reconcile that outcome before retrying or switching. Our example only prepares a draft and grants neither model permission to send it.
Evidence and limits
- The workflow, dependencies and rehearsal conditions are invented. We have not run the models or performed the outage exercise described here.
- The worksheet is a proposed method. It establishes no availability, recovery time, cost saving, provider ranking or legal compliance.
- A passing rehearsal would support only the tested configuration and failure. Changes to models, access or shared services need their own review.
Choose one task and draw its current route with the person who operates it. Mark the interruption the proposed second provider is meant to handle. If you cannot yet explain how the work would finish in that case, the next useful step is to specify and rehearse that path.
