Triage your failures

The Triage view answers the question you have after every deploy: which failures should we work on first? It places each active failure cluster by how many sessions it reaches and how fast it is growing. It marks which clusters your agent can fix, and it ranks the fixable ones in a Fix queue. It also counts silent failures, meaning sessions that returned ok but gave the user a broken answer.

At a glance

The Failure Modes page has four views: Failure Modes (the cluster cards), Triage (this page), Atlas (see Failure atlas) and Issues & Taxonomy (see Issues & failure modes). Old links to the retired Scatter view open Triage.

A worked example

Take an e-commerce support agent. In the last 7 days it handled 4,812 sessions. One cluster, check_inventory 5xx/504 cascade, collects sessions where the inventory service returned 504s and the agent carried on anyway. It had 104 sessions in the previous 7 days and 286 in this window. 121 of those 286 sessions ended with status ok, but a judge marked the answer as failed, because the agent told customers an item was in stock without checking.

Here is how that cluster appears:

On the matrix it sits high and to the right, in the FIX NOW quadrant. It gets a solid ring because the agent can fix it, and a centre dot because more than 25% of its sessions are silent failures.

Two other clusters in the same window show why it tops the Fix queue:

The new cluster is smaller but gets the maximum growth weight. The refund cluster is chronic: it is large but barely growing. The inventory cluster is large, growing fast and mostly silent, so it ranks first. The formula is described in How the fix queue is ranked.

Read the triage view

The summary cards

The triage matrix

Each bubble is one active cluster with sessions in the window. Its size is the session count.

• Horizontal position is growth: the change in sessions compared with the previous window. The axis runs from −60% to +320%. Clusters beyond either end are drawn at the edge.
• Vertical position is share: the cluster’s sessions as a percentage of all the agent’s sessions in the window.
• The NEW lane on the right holds clusters first seen in this window with no sessions in the previous one. They have no growth figure, so they get a lane of their own and are not placed on the growth axis. With All time there is no previous window, so no cluster has a growth figure or counts as new: every bubble sits on the 0% line and the NEW lane stays empty.

Two dashed lines split the chart into four quadrants. The vertical line is 0% change. The horizontal line is the median share of the clusters on the chart, so “large” and “small” are relative to your other clusters in the same window. The FIX NOW quadrant is shaded.

The ring around a bubble shows fixability:

A centre dot means at least 25% of the cluster’s sessions are silent failures. A green trail from the cluster’s previous position means a fix was merged and Neens is watching whether it holds (see Fix outcomes).

Hover a bubble for its numbers. Click it, or focus it with Tab and press Enter, to open its panel. The bucket chips above the matrix (Tool failures, Timeouts and so on) filter the matrix, the queue and the sessions table to one bucket.

The fix queue

The panel next to the matrix has three parts:

• Fix queue: agent-fixable and needs-triage clusters, ranked by priority. Each row shows its inputs (growth, sessions, silent count), so you can see why it ranks where it does.
• Fix merged · watching: clusters with a merged fix whose effect Neens is still measuring. They leave the queue while they are watched.
• Not fixable in this agent: downstream-service, guardrail and other not-fixable clusters.

The cluster panel

Clicking a cluster shows its session count (and the previous window’s), its share, its wasted spend, a sessions-over-time sparkline, and a What the user got bar that splits its sessions by outcome. It also shows the current remediation and what state it is in: proposed, in a fix run, watching after merge, verified, or regressed.

From the panel you can generate a remediation, start or view a fix run, build a regression set, or open Where it breaks → (see Where it breaks).

Silent failures and where failures end up

Click the Silent failures card to open Where failures end up. It is a flow chart from each failure bucket on the left to what the user got on the right: Errored, Silent failure, Recovered or Not confirmed. Band widths are session counts. Click a band to filter the sessions table to that bucket and outcome. Click an outcome on the right to filter by outcome only.

The sessions table

The sessions table under the view follows the window and filters above it. On this page it includes an Outcome column by default. When you filter it to Silent failure, a banner offers Add to dataset so you can collect those sessions for a regression set. To get the same filter from the API, pass outcome=errored|silent|recovered|unconfirmed to GET /sessions. Each returned row also carries its outcome.

How-tos

Triage a new cluster after a deploy

Find silent failures and add them to a dataset

How is it defined

How outcomes are defined

Every failing session gets exactly one outcome:

A silent failure can have no errored step at all, for example when the tool calls succeeded and the final answer was wrong. Not confirmed sessions often become silent failures once a judge runs over them or someone reviews them. See Judges and Review & annotations.

How growth is defined

Growth compares the cluster’s sessions in this window (n) with the equal-length window just before it (prev):

growth = (n − prev) ÷ max(prev, 5) × 100

The floor of 5 previous sessions keeps tiny clusters from swinging wildly. A cluster that goes from 1 session to 4 shows +60% ((4 − 1) ÷ 5), not +300%.

A cluster is new when it was first seen inside this window and had no sessions in the previous one. New clusters have no growth figure and go in the NEW lane. A cluster that existed before but had no sessions in the previous window is not new; its growth uses the floor, so 8 sessions from 0 shows +160%.

How fixability is defined

Fixability comes from the cluster’s top remediation:

Regenerating or editing the remediation changes the cluster’s fixability. See Remediations.

How the fix queue is ranked

priority = share × growth weight × (1 + silent share)

share          = cluster sessions ÷ all sessions in the window, as a percentage
growth weight  = 3                              for a new cluster
               = 1 + min(2, max(0, growth) ÷ 100) otherwise
silent share   = silent failures ÷ cluster sessions

So growth can at most triple a cluster’s weight, and shrinking clusters are never pushed below their share. A cluster where every session is a silent failure counts double. The same score orders the queue in the UI and in the API response, so both always agree.

Troubleshooting