When a web service breaks, it usually breaks loudly: a 500, a stack trace, a page. Agents are different. A model that picks the wrong tool, loops on a retry or runs out of context still returns something that looks like an answer. Nothing alerts. The user just gets a worse result.
The three quiet failures
After reviewing thousands of incidents across Sigil customers, almost every silent failure fell into one of three buckets:
Tool drift: the agent calls a tool with arguments that are valid but wrong, like searching the right table for the wrong customer.
Context loss: a long run silently drops early instructions once the context window fills up.
Soft timeouts: a slow tool returns partial data and the agent carries on as if it were complete.
Make every step observable
You can’t alert on what you don’t record. Sigil traces every model call and tool call as a span, with inputs, outputs, tokens and cost. That turns “the answer felt off” into a timeline you can inspect step by step.
ts
sigil.on("span.end", (span) => { if (span.tool && span.output?.partial) { span.flag("partial_tool_result") } })
Alert on behaviour, not just errors
The most useful alerts we see aren’t on exceptions. They’re on shape: runs that take twice as many steps as usual, tools called with empty arguments, or answers produced without the tool that should have been used.
The scariest incidents are the ones with a green dashboard.
Start with one behavioural alert on your most important agent. You’ll be surprised how often it fires.
