Start by separating failure types
An AI automation can fail before a model receives a request, while it is generating a response, or after it has responded but before your application receives the result. Those cases can look similar to a user, yet they require different handling. A timeout means your application stopped waiting; it does not necessarily mean the model stopped working or that no response was produced.
Classify errors before deciding what to do. A brief network interruption, a temporary service error or a rate limit may justify another attempt. An invalid request, missing permission or unsupported input usually will not improve if you send it again unchanged. A response that arrives but fails your format checks is another category: you may be able to repair or regenerate it, but you should not treat it as a connection failure.
Set a deadline for the whole user-facing task, not just a timeout for each model call. Otherwise, several individually reasonable attempts can add up to an unreasonable wait. Decide in advance how much time the workflow can spend trying to produce an answer and what it will do when that budget runs out.
Make retries bounded and selective
A useful retry policy answers four questions: which failures qualify, how many additional attempts are allowed, how long to wait between them, and when the overall deadline takes priority. Keep the number of attempts small and increase the delay between them rather than immediately repeating a request. A little randomness in the delay can help prevent many workers from retrying at the same moment.
If a service provides a specific instruction about when to retry, account for it within your task deadline. Do not retry indefinitely to satisfy one request. Repeated calls can increase cost, worsen congestion and leave users waiting without improving the result. For malformed input or an authorization error, surface a useful error or route the task for correction instead.
Before retrying, consider whether the operation is safe to repeat. Generating text may produce a different answer on a second attempt, but a workflow that also sends an email, updates a record or charges a customer must not repeat that action blindly. Keep model generation separate from irreversible steps. Where supported, use a stable task identifier or idempotency key, and check whether an action already completed before doing it again.
Design fallbacks around the task
A fallback is not simply a second model call. It is an alternative way to meet the user's underlying need when the preferred path cannot finish. For a document-summary tool, that might mean switching to a shorter input, producing a basic extractive summary from already available text, or asking the user to try again. For a draft-generation tool, saving the user's inputs and returning a clear status may be better than silently substituting an unsuitable result.
Choose fallback paths according to the failure. A smaller request may help when processing a long input is the problem, but it will not fix invalid credentials. A different model or provider might help during a temporary outage, but its output may vary in format, quality, privacy handling or cost. Test whether each fallback meets the same minimum requirements before treating the paths as interchangeable.
Make the change visible when it matters. If a fallback offers a shorter answer or skips a nonessential step, say so. If the task requires verification, structured fields or a particular level of completeness, do not present an incomplete fallback as if it passed those checks. Sometimes the responsible fallback is a plain failure message with the user's work preserved.
Validate results before continuing
A successful network response is not necessarily a successful workflow result. Check that the output contains the fields you need, fits the expected format and does not violate basic task constraints. If a model returns unusable structured data, one targeted repair attempt may be more appropriate than rerunning the entire workflow. Stop if the result still fails the checks needed for the next step.
Treat uncertain completion as its own state. After a timeout, do not assume that a previous request failed if the service or your application might still be processing it. Before repeating a task that can trigger downstream effects, look for a recorded result using the task identifier. This is especially important when a model response feeds an automated action rather than a draft for human review.
Keep a human-review path for consequential or ambiguous cases. Automation can detect missing fields and duplicate task identifiers, but it cannot guarantee that every plausible-looking answer is correct. A workflow should be able to pause instead of forcing a result through validation.
Test and observe the full failure path
Test the unhappy paths deliberately: a slow response, a rate limit, a temporary error, an invalid request, malformed output and a timeout just before an action completes. Confirm that attempts stop at the deadline, users receive an understandable status, and downstream actions happen no more than intended. Run these tests against each fallback as well as the primary path.
Record enough information to diagnose problems: a task identifier, failure category, attempt count, elapsed time, fallback chosen and final outcome. Avoid logging sensitive prompts or model responses by default; collect only what your team needs and protect it appropriately. Reviewing these records can show whether retries are helping or merely delaying failures. The goal is not to hide every error, but to make each outcome predictable, safe and understandable.
