To set a separate retry policy for LLM calls, pass [`RunOptions`](https://restatedev.github.io/sdk-typescript/types/_restatedev_restate-sdk.RunOptions.html) to the `durableCalls` middleware: ```typescript errorhandling/fail-on-terminal-tool-agent.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/vercel-ai/tour-of-agents/src/errorhandling/fail-on-terminal-tool-agent.ts#max_attempts_example"} theme={null} const model = wrapLanguageModel({ model: openai("gpt-5.4"), middleware: durableCalls(ctx, { maxRetryAttempts: 3 }), }); ``` If you set a maximum number of retry attempts, Restate will still go through the AI SDK's `maxRetries` for each attempt, so the two limits multiply (e.g. `maxRetryAttempts`: 3 × `maxRetries`: 2 = up to 6 attempts). Once Restate's retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly. To set a separate retry policy for LLM calls, pass [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37) to `DurableRunner.run`: ```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/openai-agents/tour-of-agents/app/error_handling.py#handle"} theme={null} @agent_service.handler() async def run(_ctx: restate.Context, req: WeatherPrompt) -> str: try: run_opts = RunOptions( max_attempts=3, initial_retry_interval=timedelta(seconds=2) ) result = await DurableRunner.run(agent, req.message, run_options=run_opts) except restate.TerminalError as e: # Handle terminal errors gracefully return f"The agent couldn't complete the request: {e.message}" return result.final_output ``` Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly. To set a separate retry policy for LLM calls, pass [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37) to the Restate plugin when activating it for your ADK App: ```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/google-adk/tour-of-agents/app/error_handling.py#retries"} theme={null} run_options = RunOptions(max_attempts=3, initial_retry_interval=timedelta(seconds=1)) app = App( name=APP_NAME, root_agent=agent, plugins=[RestatePlugin(run_options=run_options)], ) ``` Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly. To set a separate retry policy for LLM calls, pass [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37) to `RestateAgent`: ```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/pydantic-ai/tour-of-agents/app/error_handling.py#retries"} theme={null} restate_agent = RestateAgent( agent, run_options=RunOptions(max_attempts=3, initial_retry_interval=timedelta(seconds=2)), ) ``` Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly. Restate's `RestateMiddleware` lets you specify the retry behavior for LLM calls via `RunOptions`: ```python error_handling.py theme={null} agent = create_agent( model=init_chat_model("openai:gpt-4o-mini"), tools=[get_weather], middleware=[RestateMiddleware(run_options=RunOptions(max_attempts=3))], ) ``` By default, the middleware retries indefinitely with exponential backoff. Once Restate's retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. To set a separate retry policy for LLM calls, pass [`RunOptions`](https://restatedev.github.io/sdk-typescript/types/_restatedev_restate-sdk.RunOptions.html) to `ctx.run()`: ```typescript theme={null} // Retries up to 3 times with exponential backoff const result = await ctx.run( "LLM call", async () => llmCall(messages, tools), { maxRetryAttempts: 3 } ); ``` Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly. To set a separate retry policy for LLM calls, pass [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37) to `ctx.run_typed()`: ```python theme={null} # Retries up to 3 times with exponential backoff result = await ctx.run_typed( "LLM call", llm_call, RunOptions(max_attempts=3), messages=messages, tools=tools, ) ``` Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly.

By default, the Vercel AI SDK converts any errors in tool executions into a message to the LLM, and the agent decides how to proceed. This is often desirable, as the LLM can decide to use a different tool or provide a fallback answer. When you wrap external calls in Restate Context actions like `ctx.run`, Restate retries transient errors within the Context action before the result reaches the agent. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions: ```typescript {"CODE_LOAD::ts/src/tour/agents/inline-tool-errors.ts#here"} theme={null} // Without ctx.run - error goes straight to agent async function myTool() { const result = await fetch("/api/data"); // Might fail due to network // If this fails, agent gets the error immediately } // With ctx.run - Restate handles retries async function myToolWithRestate(ctx: restate.Context) { const result = await ctx.run("fetch-data", () => fetch("/api/data")); // Network failures get retried automatically // Only terminal errors reach the AI } ``` Restate then retries the whole invocation according to the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults). Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues. By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults). Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues. By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults). Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues. By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults). Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues. By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults). Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues. By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).

If you do run actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://restatedev.github.io/sdk-typescript/types/_restatedev_restate-sdk.RunOptions.html): ```ts {"CODE_LOAD::ts/src/ai/guides/errorhandling/error_handling.ts#retries"} theme={null} const result = await ctx.run( "fetch-data", () => fetch("/api/data"), { maxRetryAttempts: 3 } ); ``` See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error. If you do run actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37): ```python {"CODE_LOAD::python/src/ai/error_handling.py#retries"} theme={null} result = await restate_context().run_typed( "fetch data", fetch_data, RunOptions(max_attempts=3), req=req, ) ``` See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error. If you do run actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37): ```python {"CODE_LOAD::python/src/ai/error_handling.py#retries"} theme={null} result = await restate_context().run_typed( "fetch data", fetch_data, RunOptions(max_attempts=3), req=req, ) ``` See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error. If you do run actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37): ```python {"CODE_LOAD::python/src/ai/error_handling.py#retries"} theme={null} result = await restate_context().run_typed( "fetch data", fetch_data, RunOptions(max_attempts=3), req=req, ) ``` See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error. If you do `ctx.run` actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://restatedev.github.io/sdk-typescript/types/_restatedev_restate-sdk.RunOptions.html): ```ts {"CODE_LOAD::ts/src/ai/guides/errorhandling/error_handling.ts#retries"} theme={null} const result = await ctx.run( "fetch-data", () => fetch("/api/data"), { maxRetryAttempts: 3 } ); ``` See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error. For `ctx.run_typed` actions specifically, you can override the default retry policy by passing [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37): ```python {"CODE_LOAD::python/src/ai/error_handling.py#retries"} theme={null} result = await restate_context().run_typed( "fetch data", fetch_data, RunOptions(max_attempts=3), req=req, ) ``` See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error.

```typescript {"CODE_LOAD::ts/src/tour/agents/terminal_error.ts#terminal_error"} theme={null} throw new TerminalError("This tool is not allowed to run for this input."); ``` By default, Vercel AI converts the terminal error into a message to the LLM, and the agent decides how to proceed. If you want to treat terminal tool errors as permanent failures and stop the agent instead, the Restate middleware provides two utilities: **To fail the agent on terminal tool errors**, rethrow the error in `onStepFinish`: ```typescript errorhandling/fail-on-terminal-tool-agent.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/vercel-ai/tour-of-agents/src/errorhandling/fail-on-terminal-tool-agent.ts#option2"} theme={null} const { text } = await generateText({ model, tools: { getWeather: tool({ description: "Get the current weather for a given city.", inputSchema: z.object({ city: z.string() }), execute: async ({ city }) => { return await ctx.run("get weather", () => fetchWeather(city)); }, }), }, stopWhen: [stepCountIs(5)], onStepFinish: rethrowTerminalToolError, system: "You are a helpful agent that provides weather updates.", messages: [{ role: "user", content: prompt }], }); ``` To stop the agent on terminal tool errors and handle it after the agent finishes, you can use `hasTerminalToolError` in `stopWhen` and then inspect the steps for errors: ```typescript errorhandling/stop-on-terminal-tool-agent.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/vercel-ai/tour-of-agents/src/errorhandling/stop-on-terminal-tool-agent.ts#option3"} theme={null} const { steps, text } = await generateText({ model, tools: { getWeather: tool({ description: "Get the current weather for a given city.", inputSchema: z.object({ city: z.string() }), execute: async ({ city }) => { return await ctx.run("get weather", () => fetchWeather(city)); }, }), }, stopWhen: [stepCountIs(5), hasTerminalToolError], system: "You are a helpful agent that provides weather updates.", messages: [{ role: "user", content: prompt }], }); const terminalSteps = getTerminalToolSteps(steps); if (terminalSteps.length > 0) { // Do something with the terminal tool error steps } ``` ```python {"CODE_LOAD::python/src/ai/error_handling.py#terminal"} theme={null} from restate import TerminalError raise TerminalError("This tool is not allowed to run for this input.") ``` The Restate OpenAI integration raises terminal errors to your handler, where you can catch and handle them: ```python {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/openai-agents/tour-of-agents/app/error_handling.py#handle"} theme={null} @agent_service.handler() async def run(_ctx: restate.Context, req: WeatherPrompt) -> str: try: run_opts = RunOptions( max_attempts=3, initial_retry_interval=timedelta(seconds=2) ) result = await DurableRunner.run(agent, req.message, run_options=run_opts) except restate.TerminalError as e: # Handle terminal errors gracefully return f"The agent couldn't complete the request: {e.message}" return result.final_output ``` The OpenAI Agent SDK also allows setting `failure_error_function` to `None`, which will rethrow any error in the agent execution as-is. Also for example invalid LLM responses (e.g. tool call with invalid arguments or to a tool that doesn't exist). The error will then lead to Restate retries. Since the error isn't transient, the invocation will be paused when the retries are exhausted, and will require manual intervention. Therefore, we do not recommend using this setting and instead recommend handling these errors appropriately in your agent logic. ```python {"CODE_LOAD::python/src/ai/error_handling.py#terminal"} theme={null} from restate import TerminalError raise TerminalError("This tool is not allowed to run for this input.") ``` You can catch these terminal errors in your handler and handle them accordingly: ```python {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/google-adk/tour-of-agents/app/error_handling.py#handle"} theme={null} @agent_service.handler() async def run(ctx: restate.ObjectContext, req: WeatherPrompt) -> str | None: try: events = runner.run_async( user_id=ctx.key(), session_id=req.session_id, new_message=Content(role="user", parts=[Part.from_text(text=req.message)]), ) return await parse_agent_response(events) except TerminalError as e: # Handle the error appropriately, e.g., log it or return a default response return "Sorry, I'm unable to process your request at the moment." ``` ```python {"CODE_LOAD::python/src/ai/error_handling.py#terminal"} theme={null} from restate import TerminalError raise TerminalError("This tool is not allowed to run for this input.") ``` You can catch these terminal errors in your handler and handle them accordingly: ```python {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/pydantic-ai/tour-of-agents/app/error_handling.py#handle"} theme={null} @agent_service.handler() async def run(_ctx: restate.Context, req: WeatherPrompt) -> str: try: result = await restate_agent.run(req.message) except TerminalError as e: # Handle terminal errors gracefully return f"The agent couldn't complete the request: {e.message}" return result.output ``` When agent tools use Restate Context actions like `ctx.run`, Restate automatically retries transient errors in these operations. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions. For example, wrapping a tool call in `restate_context().run_typed()` makes it durable with automatic retries: ```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/langchain-python/tour-of-agents/app/error_handling.py#here"} theme={null} @tool async def get_weather(city: WeatherRequest) -> WeatherResponse: """Get the current weather for a given city.""" return await restate_context().run_typed( "get weather", fetch_weather, RunOptions(max_attempts=3), req=city ) ``` For errors that should not be retried, raise a terminal error: ```python theme={null} from restate import TerminalError raise TerminalError("This tool is not allowed to run for this input.") ``` Restate retries tool executions until they succeed. Terminal errors propagate past LangChain's tool-error handling back to the service handler, where you can catch them: ```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/langchain-python/tour-of-agents/app/error_handling.py#handle"} theme={null} try: result = await agent.ainvoke({"messages": req.message}) except restate.TerminalError as e: return f"The agent couldn't complete the request: {e.message}" ``` ```typescript {"CODE_LOAD::ts/src/tour/agents/terminal_error.ts#terminal_error"} theme={null} throw new TerminalError("This tool is not allowed to run for this input."); ``` You can catch and handle terminal errors in your agent logic if needed. ```python {"CODE_LOAD::python/src/ai/error_handling.py#terminal"} theme={null} from restate import TerminalError raise TerminalError("This tool is not allowed to run for this input.") ``` You can catch and handle terminal errors in your agent logic if needed.