Retries & Error Handling

Implement robust error handling and retry strategies for reliable agents.

Restate automatically retries failures of your agents until they succeed. But LLM calls are costly, so you might want to configure retry behavior to fit your use case and to avoid retrying errors that cannot heal.

Restate distinguishes between two types of errors:

• Transient errors: Temporary issues like network failures or rate limits. Restate automatically retries these until they succeed or the retry policy is exhausted.
• Terminal errors: Permanent failures like invalid input or business rule violations. Restate does not retry these. The invocation fails permanently. You can catch these errors and handle them gracefully.

Retrying LLM calls

LLM API calls can suffer from transient failures (rate limits, network issues, provider outages). Restate retries failed LLM calls so your agents recover automatically.

Default behavior

Vercel AI

The Vercel AI SDK and the Restate middleware each have their own retry layer, and they compose.

The Vercel AI SDK does the first layer of retries based on what is set for maxRetries on generateText (default: 2) . Once those are exhausted, the AI SDK throws an error.

Restate then takes over and retries the invocation. Each Restate retry replays the call, which goes through maxRetries Vercel AI SDK attempts again.

By default, Restate’s retries follow the policy configured at the service or handler level, or otherwise the Restate server’s default policy. Restate will go through a limited set of retries with exponential backoff (see default policy), after which the invocation will be paused. This gives you time to fix the issue, and then resume the invocation.

OpenAI Agents

By default, DurableRunner.run retries LLM calls according to the policy configured at the service or handler level, or otherwise the Restate server’s default policy. Restate will go through a limited set of retries with exponential backoff (see default policy), after which the invocation will be paused. This gives you time to fix the issue, and then resume the invocation.

Google ADK

By default, the RestatePlugin retries LLM calls according to the policy configured at the service or handler level, or otherwise the Restate server’s default policy. Restate will go through a limited set of retries with exponential backoff (see default policy), after which the invocation will be paused. This gives you time to fix the issue, and then resume the invocation.

Pydantic AI

By default, RestateAgent retries LLM calls according to the policy configured at the service or handler level, or otherwise the Restate server’s default policy. Restate will go through a limited set of retries with exponential backoff (see default policy), after which the invocation will be paused. This gives you time to fix the issue, and then resume the invocation.

Restate TS

When you wrap LLM calls in ctx.run(), Restate retries them according to the policy configured at the service or handler level, or otherwise the Restate server’s default policy. Restate will go through a limited set of retries with exponential backoff (see default policy), after which the invocation will be paused. This gives you time to fix the issue, and then resume the invocation.

Restate Py

When you wrap LLM calls in ctx.run_typed(), Restate retries them according to the policy configured at the service or handler level, or otherwise the Restate server’s default policy. Restate will go through a limited set of retries with exponential backoff (see default policy), after which the invocation will be paused. This gives you time to fix the issue, and then resume the invocation.

Setting a retry policy

Vercel AI

To set a separate retry policy for LLM calls, pass RunOptions to the durableCalls middleware:

errorhandling/fail-on-terminal-tool-agent.ts

const model = wrapLanguageModel({
  model: openai("gpt-5.4"),
  middleware: durableCalls(ctx, { maxRetryAttempts: 3 }),
});

If you set a maximum number of retry attempts, Restate will still go through the AI SDK’s maxRetries for each attempt, so the two limits multiply (e.g. maxRetryAttempts: 3 × maxRetries: 2 = up to 6 attempts).

Once Restate’s retries are exhausted, the invocation fails with a TerminalError and won’t be retried further. You can catch the Terminal Error in your handler and act accordingly.

OpenAI Agents

To set a separate retry policy for LLM calls, pass RunOptions to DurableRunner.run:

error_handling.py

@agent_service.handler()
async def run(_ctx: restate.Context, req: WeatherPrompt) -> str:
    try:
        run_opts = RunOptions(
            max_attempts=3, initial_retry_interval=timedelta(seconds=2)
        )
        result = await DurableRunner.run(agent, req.message, run_options=run_opts)
    except restate.TerminalError as e:
        # Handle terminal errors gracefully
        return f"The agent couldn't complete the request: {e.message}"

    return result.final_output

Once these retries are exhausted, the invocation fails with a TerminalError and won’t be retried further. You can catch the Terminal Error in your handler and act accordingly.

Google ADK

To set a separate retry policy for LLM calls, pass RunOptions to the Restate plugin when activating it for your ADK App:

error_handling.py

run_options = RunOptions(max_attempts=3, initial_retry_interval=timedelta(seconds=1))
app = App(
    name=APP_NAME,
    root_agent=agent,
    plugins=[RestatePlugin(run_options=run_options)],
)

Once these retries are exhausted, the invocation fails with a TerminalError and won’t be retried further. You can catch the Terminal Error in your handler and act accordingly.

Pydantic AI

To set a separate retry policy for LLM calls, pass RunOptions to RestateAgent:

error_handling.py

restate_agent = RestateAgent(
    agent,
    run_options=RunOptions(max_attempts=3, initial_retry_interval=timedelta(seconds=2)),
)

Once these retries are exhausted, the invocation fails with a TerminalError and won’t be retried further. You can catch the Terminal Error in your handler and act accordingly.

LangChain

Restate’s RestateMiddleware lets you specify the retry behavior for LLM calls via RunOptions:

error_handling.py

agent = create_agent(
    model=init_chat_model("openai:gpt-4o-mini"),
    tools=[get_weather],
    middleware=[RestateMiddleware(run_options=RunOptions(max_attempts=3))],
)

By default, the middleware retries indefinitely with exponential backoff. Once Restate’s retries are exhausted, the invocation fails with a TerminalError and won’t be retried further.

Restate TS

To set a separate retry policy for LLM calls, pass RunOptions to ctx.run():

// Retries up to 3 times with exponential backoff
const result = await ctx.run(
  "LLM call",
  async () => llmCall(messages, tools),
  { maxRetryAttempts: 3 }
);

Once these retries are exhausted, the invocation fails with a TerminalError and won’t be retried further. You can catch the Terminal Error in your handler and act accordingly.

Restate Py

To set a separate retry policy for LLM calls, pass RunOptions to ctx.run_typed():

# Retries up to 3 times with exponential backoff
result = await ctx.run_typed(
    "LLM call",
    llm_call,
    RunOptions(max_attempts=3),
    messages=messages,
    tools=tools,
)

Once these retries are exhausted, the invocation fails with a TerminalError and won’t be retried further. You can catch the Terminal Error in your handler and act accordingly.

Tool execution errors

Restate makes tool execution resilient by retrying transient errors and propagating terminal ones.

Transient errors

Vercel AI

By default, the Vercel AI SDK converts any errors in tool executions into a message to the LLM, and the agent decides how to proceed. This is often desirable, as the LLM can decide to use a different tool or provide a fallback answer.

When you wrap external calls in Restate Context actions like ctx.run, Restate retries transient errors within the Context action before the result reaches the agent. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions:

// Without ctx.run - error goes straight to agent
async function myTool() {
  const result = await fetch("/api/data"); // Might fail due to network
  // If this fails, agent gets the error immediately
}

// With ctx.run - Restate handles retries
async function myToolWithRestate(ctx: restate.Context) {
  const result = await ctx.run("fetch-data", () => fetch("/api/data"));
  // Network failures get retried automatically
  // Only terminal errors reach the AI
}

Restate then retries the whole invocation according to the policy configured at the service or handler level, or otherwise the Restate server’s default policy.

OpenAI Agents

Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.

By default, it uses the policy configured at the service or handler level, or otherwise the Restate server’s default policy.

Google ADK

Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.

By default, it uses the policy configured at the service or handler level, or otherwise the Restate server’s default policy.

Pydantic AI

Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.

By default, it uses the policy configured at the service or handler level, or otherwise the Restate server’s default policy.

Restate TS

Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.

By default, it uses the policy configured at the service or handler level, or otherwise the Restate server’s default policy.

Restate Py

Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.

By default, it uses the policy configured at the service or handler level, or otherwise the Restate server’s default policy.

Setting a retry policy on run actions

Vercel AI

If you do run actions in your tools, you can override the default retry policy by passing RunOptions:

const result = await ctx.run(
    "fetch-data",
    () => fetch("/api/data"),
    { maxRetryAttempts: 3 }
);

See custom retry policies for more options. When retries are exhausted, the tool will fail with a Terminal Error.

OpenAI Agents

If you do run actions in your tools, you can override the default retry policy by passing RunOptions:

result = await restate_context().run_typed(
    "fetch data",
    fetch_data,
    RunOptions(max_attempts=3),
    req=req,
)

See custom retry policies for more options. When retries are exhausted, the tool will fail with a Terminal Error.

Google ADK

If you do run actions in your tools, you can override the default retry policy by passing RunOptions:

result = await restate_context().run_typed(
    "fetch data",
    fetch_data,
    RunOptions(max_attempts=3),
    req=req,
)

See custom retry policies for more options. When retries are exhausted, the tool will fail with a Terminal Error.

Pydantic AI

If you do run actions in your tools, you can override the default retry policy by passing RunOptions:

result = await restate_context().run_typed(
    "fetch data",
    fetch_data,
    RunOptions(max_attempts=3),
    req=req,
)

See custom retry policies for more options. When retries are exhausted, the tool will fail with a Terminal Error.

Restate TS

If you do ctx.run actions in your tools, you can override the default retry policy by passing RunOptions:

const result = await ctx.run(
    "fetch-data",
    () => fetch("/api/data"),
    { maxRetryAttempts: 3 }
);

See custom retry policies for more options. When retries are exhausted, the tool will fail with a Terminal Error.

Restate Py

For ctx.run_typed actions specifically, you can override the default retry policy by passing RunOptions:

result = await restate_context().run_typed(
    "fetch data",
    fetch_data,
    RunOptions(max_attempts=3),
    req=req,
)

See custom retry policies for more options. When retries are exhausted, the tool will fail with a Terminal Error.

Terminal errors

For errors that should not be retried (invalid input, business rule violations, resource not found), use a TerminalError in your tool. Restate does not retry these:

Vercel AI

throw new TerminalError("This tool is not allowed to run for this input.");

By default, Vercel AI converts the terminal error into a message to the LLM, and the agent decides how to proceed.

If you want to treat terminal tool errors as permanent failures and stop the agent instead, the Restate middleware provides two utilities:

Fail the agent on terminal tool errors

To fail the agent on terminal tool errors, rethrow the error in onStepFinish:

errorhandling/fail-on-terminal-tool-agent.ts

const { text } = await generateText({
  model,
  tools: {
    getWeather: tool({
      description: "Get the current weather for a given city.",
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => {
        return await ctx.run("get weather", () => fetchWeather(city));
      },
    }),
  },
  stopWhen: [stepCountIs(5)],
  onStepFinish: rethrowTerminalToolError,
  system: "You are a helpful agent that provides weather updates.",
  messages: [{ role: "user", content: prompt }],
});

Stop the agent on terminal tool errors

To stop the agent on terminal tool errors and handle it after the agent finishes, you can use hasTerminalToolError in stopWhen and then inspect the steps for errors:

errorhandling/stop-on-terminal-tool-agent.ts

const { steps, text } = await generateText({
  model,
  tools: {
    getWeather: tool({
      description: "Get the current weather for a given city.",
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => {
        return await ctx.run("get weather", () => fetchWeather(city));
      },
    }),
  },
  stopWhen: [stepCountIs(5), hasTerminalToolError],
  system: "You are a helpful agent that provides weather updates.",
  messages: [{ role: "user", content: prompt }],
});

const terminalSteps = getTerminalToolSteps(steps);
if (terminalSteps.length > 0) {
  // Do something with the terminal tool error steps
}

OpenAI Agents

from restate import TerminalError

raise TerminalError("This tool is not allowed to run for this input.")

The Restate OpenAI integration raises terminal errors to your handler, where you can catch and handle them:

@agent_service.handler()
async def run(_ctx: restate.Context, req: WeatherPrompt) -> str:
    try:
        run_opts = RunOptions(
            max_attempts=3, initial_retry_interval=timedelta(seconds=2)
        )
        result = await DurableRunner.run(agent, req.message, run_options=run_opts)
    except restate.TerminalError as e:
        # Handle terminal errors gracefully
        return f"The agent couldn't complete the request: {e.message}"

    return result.final_output

Setting `failure_error_function` to `None`

The OpenAI Agent SDK also allows setting failure_error_function to None, which will rethrow any error in the agent execution as-is. Also for example invalid LLM responses (e.g. tool call with invalid arguments or to a tool that doesn’t exist). The error will then lead to Restate retries. Since the error isn’t transient, the invocation will be paused when the retries are exhausted, and will require manual intervention. Therefore, we do not recommend using this setting and instead recommend handling these errors appropriately in your agent logic.

Google ADK

from restate import TerminalError

raise TerminalError("This tool is not allowed to run for this input.")

You can catch these terminal errors in your handler and handle them accordingly:

@agent_service.handler()
async def run(ctx: restate.ObjectContext, req: WeatherPrompt) -> str | None:
    try:
        events = runner.run_async(
            user_id=ctx.key(),
            session_id=req.session_id,
            new_message=Content(role="user", parts=[Part.from_text(text=req.message)]),
        )
        return await parse_agent_response(events)
    except TerminalError as e:
        # Handle the error appropriately, e.g., log it or return a default response
        return "Sorry, I'm unable to process your request at the moment."

Pydantic AI

from restate import TerminalError

raise TerminalError("This tool is not allowed to run for this input.")

You can catch these terminal errors in your handler and handle them accordingly:

@agent_service.handler()
async def run(_ctx: restate.Context, req: WeatherPrompt) -> str:
    try:
        result = await restate_agent.run(req.message)
    except TerminalError as e:
        # Handle terminal errors gracefully
        return f"The agent couldn't complete the request: {e.message}"
    return result.output

LangChain

When agent tools use Restate Context actions like ctx.run, Restate automatically retries transient errors in these operations. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions.

For example, wrapping a tool call in restate_context().run_typed() makes it durable with automatic retries:

error_handling.py

@tool
async def get_weather(city: WeatherRequest) -> WeatherResponse:
    """Get the current weather for a given city."""
    return await restate_context().run_typed(
        "get weather", fetch_weather, RunOptions(max_attempts=3), req=city
    )

For errors that should not be retried, raise a terminal error:

from restate import TerminalError

raise TerminalError("This tool is not allowed to run for this input.")

Restate retries tool executions until they succeed. Terminal errors propagate past LangChain’s tool-error handling back to the service handler, where you can catch them:

error_handling.py

try:
    result = await agent.ainvoke({"messages": req.message})
except restate.TerminalError as e:
    return f"The agent couldn't complete the request: {e.message}"

Restate TS

throw new TerminalError("This tool is not allowed to run for this input.");

You can catch and handle terminal errors in your agent logic if needed.

Restate Py

from restate import TerminalError

raise TerminalError("This tool is not allowed to run for this input.")

You can catch and handle terminal errors in your agent logic if needed.

To learn more about error handling with Restate, consult the error handling guide.

Combining with rollback

For multi-step agent workflows where steps have side effects (bookings, payments, emails), combine terminal errors with compensation/rollback patterns to undo completed work before finishing.