Tool-call timeouts set by survival analysis
Your 30-second timeout is a number somebody typed
Most tool calls finish in a second. Some never finish. The timeout in the config decides how long every hung call costs you, and nobody knows what cutting it would break. Timeout computes the second at which waiting stops paying.
The problem
Every hung call costs the full timeout, and nobody dares lower it
The agent calls search_orders five thousand times a week. Most calls return in about a second. A few percent hang on a dead connection and sit there until the 30-second timeout fires, then retry. Cutting the timeout to five seconds would save most of that wait, but it might also kill calls that were about to return, and the p99 on the APM dashboard cannot tell you which, because it was computed only on the calls that came back.
The insight
The right question is conditional: given it has waited this long, will one more second help
A timeout is a decision made while a call is still running. The number that decides it is not the p99. It is the chance that a call still running at t seconds completes in the next second. That is a survival question, and the hangs are censored observations, not missing data. Kaplan-Meier uses them properly. Where the conditional completion probability drops under a floor, waiting stops paying, and that second is the timeout. Everything past it is money spent on calls that were never coming back.
Kaplan-Meier on call durations right-censored at the configured timeout, the conditional completion probability S(t+1)/S(t) per second, and a cut at the first second where its Wilson upper bound falls below a completion floor.
How it works
Four steps, no data science team
Durations from your tracing backend, with the calls that hit the timeout kept as censored rather than dropped.
Kaplan-Meier over every call, so the hangs push the plateau up without being mistaken for slow completions.
Per second, the probability a still-running call completes in the next one. The timeout is where that goes under the floor.
A per-tool timeout with the false-kill rate and the seconds saved per hang, as a config diff you can read.
Who it is for
The engineer who owns the agent's latency budget
Teams running agents that make real tool calls at volume, where the timeout is a line in a config file that someone copied from another config file.
Pricing
- –Completion curve
- –Recommended timeout
- –False-kill estimate
- –Per-tool policies
- –Config diffs
- –Weekly drift check
- –Retry budget
- –Per-segment curves
- –Self-hosted
- –SSO
- –Gateway export
Competition
What exists, and what it does not do
| Who | What they do | The gap |
|---|---|---|
| tenacity, retry libraries | Retry a failed call with backoff. | They run after the timeout fires. They have no opinion on what the timeout should be, which is the number that sets the cost of every retry. |
| Gateway timeouts (Envoy, LiteLLM) | Enforce a per-route timeout at the proxy. | Enforcement, not estimation. The value in the route config is still typed in by hand. |
| APM p99 dashboards | Show the latency distribution of completed calls. | Computed only on calls that returned. The hangs are the whole problem and they are not in the histogram. |
| Type 30 into the config | What everyone does today. | Every hung call costs 30 seconds and a retry. Nobody knows what cutting it would break, so nobody cuts it. |
The method assumes a call that hung would not otherwise have been an unusually slow completion. If hangs and slow completions share a cause, censoring is informative and the curve understates the tail, so the demo shows the plateau diagnostic: completions between ten seconds and the timeout should be near zero. Any APM vendor could add this to a latency panel in a sprint, and a gateway could ship it as an auto-tune flag. It fails if teams keep treating the timeout as a number nobody owns rather than a cost somebody pays.
Market
Priced against the latency budget of production agents, which is already a line item
Every team with an agent making tool calls in production has at least one timeout in a config. Three thousand teams at the Team tier is $12.6M ARR. The Scale tier is priced against the retry spend it removes.