Business KPIs
Nobody outside the engineering team asks how many traces the agent emitted. They ask what share of cases the agent handled on its own, how long a case takes to resolve, and what a case costs to run. Those are the numbers that end up in a board deck, and they are measured per case — one customer conversation — not per model call.
The Business KPIs page is where those numbers live. It sits in your agent’s left nav, and it gathers everything behind a business number onto one page: the KPIs themselves, the definitions that decide what “escalated” means for your data, the outcome feeds that supply the evidence, and the custom measures you add in your own vocabulary.
Every business KPI arrives with two things attached: its provenance and its coverage. Provenance says whether the number came from your system of record, from your agent’s own claim, from a deterministic Neens computation, or from an LLM’s opinion. Coverage says what fraction of the cases in the window actually had a signal. A containment rate over 12% of your cases is a very different fact from the same number over 95% — and the page shows you which one you’re looking at.
At a glance
| Where | The Business KPIs page in your agent’s left nav (the gauge icon). |
| Tabs | Overview · Failure impact · Definitions · Data sources · Custom measures |
| The KPIs | Tiles on this page: Containment · Resolution · Resolution time. cost_per_case is computed from your traces and lives on Cost optimization, plus dashboard widgets and alerts. |
| Grain | Case — conversation_id when the trace has one, the trace’s own id when it doesn’t. |
| Key API | GET /kpi-definitions · PUT /kpi-definitions · GET /kpi-definitions/options · GET /measures/catalogue · GET /kpis/{id}/eroding-clusters |
| Who can view | Anyone with read access to the agent sees the page and its numbers. Connecting or editing an outcome feed is admin-only, so a non-admin sees the match coverage on Data sources but not the feed management. |
| Who can change | An admin declares the definitions and connects the outcome feeds; everyone else reads them. |
| Commitments | Any catalogued measure can be promoted to a KPI — a target, a direction, an owner, a status and a review cadence — over GET/POST /kpis, GET /kpis/options and GET /kpis/summary. Promoting and committing is API-first for now, so those examples are curl. |
| History | Every active KPI is recorded once per completed day, so it has a trend: GET /kpis/{id}/history, POST /kpis/{id}/snapshot, POST /kpis/snapshot. |
| Needs an LLM? | No for the measured, emitted and derived paths. The inferred path reads transcripts with an LLM classifier on your agent’s configured connection — either the ready-made escalation/resolution/sentiment classifiers or the issue classifier’s existing labels. |
| Prerequisite | Containment and resolution time need business outcomes (or issue-classifier labels). Cost per case needs model prices and nothing else. |
The Business KPIs page
The page is one place with five tabs. The tab you’re on is in the URL, so you can link straight to one. (The time range resets to its default when you reload.)
| Tab | What it’s for |
|---|---|
| Overview | The KPI tiles for the window, a Data health strip that tells you how trustworthy they are, and the status of each definition. This is the glance a PM takes in the morning. |
| Failure impact | Pick a KPI and see the failure clusters eroding it, worst first — how far each one’s rate sits below your agent baseline, and its share of the total erosion. Clusters too thin to measure are listed apart as not enough data yet, never as 0 impact. |
| Definitions | Declare what each KPI means for your data — what counts as “escalated”, which outcome carries the resolution time. Admin-only to change; anyone can read. See Define a KPI. |
| Data sources | Connect and inspect the outcome feeds behind the measured numbers — push an outcome, run a helpdesk connector, and read the match coverage. See Business outcomes. |
| Custom measures | Define KPIs of your own — your label, your source, your denominator — that behave like the ones Neens ships. See Custom measures. |
Reading the Overview honestly: the tiles fill in over time. Today the Overview shows you your
real definitions, their provenance, and the match coverage behind them — the machinery
behind the number. The current values on the tiles populate as Neens accrues KPI snapshots for
your window, so a freshly configured agent sees — on a tile, with the real coverage figure in
the Data health strip above the tiles rather than a fabricated number. A dash means “no snapshot
yet”, never zero — and if a value could not be read, the tile says that instead, so you can tell a
missing number from a failed one. Cost per case does not wait on any of this: it is
derived straight from your traces, and you can see it today on
Cost optimization.
What a case is, and why the KPI is per case
A case is conversation_id when the trace carries one, and the trace’s own id when it doesn’t —
the same rollup the Sessions page shows and the same grain business
outcomes are keyed to. A ten-turn conversation is one
case, not ten.
That matters more than it sounds. “Contained” is a property of a conversation, not of a model call: a case where the agent answered nine turns and then handed off to a human is one escalation, not nine successes and one failure. Measuring per trace would let a chatty conversation outvote a terse one, and it would let a single case appear on both sides of the same ratio.
Two consequences worth knowing before you read a chart:
- A case is attributed to the agent and the day of its first trace inside the window. If a conversation starts on a triage agent and is picked up by a billing agent, the case counts once, against triage — which keeps a per-agent breakdown summing to the total instead of double-counting hand-offs.
- A case’s identity is the pair (agent, case key).
conversation_idis your id — a helpdesk ticket number, an upstream session id — so a staging and a production agent can carry identical ids as a matter of routine. A widget spanning both counts them as two cases, not one.
The four provenance tiers
The same tile can be backed by four very different kinds of evidence, and Neens refuses to let them look identical. The provenance is derived from where the number comes from, never chosen — so a badge can never claim a number is stronger than the evidence behind it.
| Provenance | Where the number comes from | A concrete example |
|---|---|---|
| Measured | Your system of record reported it — a business outcome from your helpdesk, CRM or warehouse. The strongest claim available. | Your Zendesk feed sends a containment outcome per ticket. Containment rate reads those facts directly. |
| Emitted | Your agent asserted it about itself, in the trace metadata. Useful, but nothing outside the agent confirmed it. | Your agent writes resolved: true onto each trace. You can measure it — badged Emitted so nobody mistakes the agent’s own word for the customer’s. |
| Derived | Neens computed it from your traces with a deterministic rule — no LLM, no external system. Reproducible from the spans. | cost_per_case — token counts on your traces multiplied by your model prices, divided by the cases in the window. Pure arithmetic over data you already send. |
| Inferred | An LLM classifier read the transcript and decided. An opinion, not a fact — a directional signal when you have nothing better yet, and one Neens calibrates against reality where it can. | You have no outcome feed, so containment reads a ready-made classifier’s label — contained vs escalated_to_human — on each sampled case. |
Emitted is not measured, and inferred is not measured. If your agent writes resolved: true
into its own metadata, that is the opinion of the same agent whose failures you are trying to find.
If an LLM reads a transcript and calls a case “escalated”, that is a guess about the past, not a
record of it. Both are perfectly reasonable to measure — and both are badged so nobody reads them as
the customer’s own confirmation.
Provenance travels with the number everywhere it renders — on the tile, in a dashboard widget, and next to an alert. A screenshot of a containment rate is only interpretable with its badge, which is exactly why the badge is never optional.
The derived tier is more than cost_per_case. A whole family of business numbers is computed
straight from your traces with no feed and no definition — turns and traces per case, case duration,
cost per turn, tool success and retry rates, span error rate, first-response latency. They have a
value on day one and are promotable to KPIs just like the ones on this page. See
Derived measures.
How to read a KPI tile
Every business tile shows more than a figure:
- The value — or
—when there’s nothing honest to show. A dash means no snapshot yet, no decided cases in the window, or unavailable for a stated reason. It never means zero, and you never see$0.00standing in for “we don’t know”. - A provenance badge — Measured, Emitted, Derived or Inferred — so you know what kind of claim the number is.
- A coverage figure — the fraction of the window’s cases the definition could actually decide. A 92% containment rate over 8% of your cases is not a 92% containment rate. On the Business KPIs Overview you read this from the Data health strip; on dashboard widgets it rides on the tile itself as a coverage badge. On an inferred tile the coverage reads sampled — a deliberately partial read, not a data gap; see reading sampled coverage.
- A partial marker on
cost_per_casewhen some models in the window have no price — the value is then a floor, not the whole spend. See Cost & model pricing.
The named KPIs
The Overview shows a tile per KPI, headed by its short name — Containment, Resolution,
Resolution time. The measure keys below are what you use on dashboards, in alerts and over the
API; cost_per_case is available in those places but is not a tile on this page.
| Measure | Label | Unit | What it is |
|---|---|---|---|
containment_rate | Containment rate | ratio | Share of decided cases the agent handled end-to-end, with no escalation. |
resolution_time_p50 | Resolution time (p50) | ms | Median time to resolve a case. |
resolution_time_p90 | Resolution time (p90) | ms | The slow tail — the 10% of cases that take longest. |
cost_per_case | Cost per case | usd | LLM spend in the window ÷ cases handled in the window. |
Containment rate — the denominator is the whole point
containment_rate = contained cases / DECIDED casesA case is decided when it carries a real signal — an outcome (or a classifier label) that says what happened. A case with no signal is undecided: it is excluded from both sides of the ratio, and reported as missing coverage.
- Only containment and escalation signals decide containment. A resolution fact does not: “the case reached a resolution” is a different question from “the agent handled it”, and a case escalated to a human and then resolved by that human is resolved and not contained. A feed that only reports resolutions therefore reports no containment coverage — the truth about that feed.
- A signal whose value can’t be read decides nothing. A
containmentfact carrying text instead of a yes/no is missing coverage, not a pass.
An unmeasured case is never counted as contained. This is not configurable. If it were, a agent with 5,000 cases and 40 outcomes would report a 99% containment rate, and everything downstream of that number would be wrong in a way nothing else in the product would catch.
A case counts as escalated if either of these is true:
- it has an escalation signal with the value you declared as “escalated” (by default: an
escalationoutcome valuedtrue); or - it has a containment signal whose value is not the value you declared as “contained” (by
default: a
containmentoutcome valued anything other thantrue).
The second arm is load-bearing. Without it, a case your own backend explicitly recorded as
not contained would read as “no escalation seen, therefore contained” — silently and
systematically generous. contained = decided AND NOT escalated.
Resolution time
Per-case duration, reported as a p50 and a p90. There are three ways to obtain it, chosen per agent on the Definitions tab:
| Mode | How the duration is obtained |
|---|---|
| From the outcome’s duration | The outcome carries the elapsed time (a resolution-time or handle-time fact your helpdesk reports). Durations are normalized to milliseconds on the way in, so this is read directly — the most accurate option when you have it. |
| From the outcome’s timestamp | Neens computes it: the earliest resolving outcome’s time minus the case’s first trace start. |
| Unavailable | There is no measured resolution instant. The measure reports unavailable — and this is forced for every classifier (inferred) definition. |
A classifier can’t produce a resolution time, and Neens won’t pretend otherwise. A classifier label is a judgement about a trace; it records no moment at which the customer’s problem was settled. Resolution time therefore reports unavailable with that reason, rather than falling back to the trace’s own duration — which measures how long the agent ran, not how long the case took, and would usually be off by hours.
An outcome dated before its case started produces a negative duration. Neens excludes it (and counts it as invalid) rather than clamping it to zero — a silently-clamped negative drags a median down and looks like an improvement. The percentile is nearest rank: p90 is the value at or below which at least 90% of cases resolved.
Cost per case
cost_per_case divides the LLM spend of the window by the cases in the same window, both computed
from one pass over the same set of cases so they can never disagree. It is the derived KPI:
pure arithmetic over your traces and your prices, no feed and no model required.
It inherits everything from cost & model pricing: a model with no
price contributes tokens but no dollars, so the ratio is a floor and carries a partial badge. A
window with zero cases returns nothing (—), never $0.00. Unlike containment and resolution time,
it needs no definition. Cost per case is not one of the Business KPIs tiles: you read it on
Cost optimization, and it is available as a dashboard widget and an
alert metric.
Coverage
Coverage is the honesty number under every business KPI: how many of the window’s cases the definition could actually decide.
The Data health strip at the top of the Overview is where you read it. It carries two stats and an as-of stamp:
| The strip shows | Meaning |
|---|---|
| KPI definitions | How many KPIs this agent has, and whether they are Configured (you declared them) or Platform defaults (you haven’t yet). |
| Outcome match rate | The share of the window’s cases that have a matched outcome, with the raw X of Y matched underneath. With no cases it reads No outcomes yet; if the figure could not be read it says so instead — an unread number is never reported as an empty one. |
| As of | When the numbers were computed — see late-arriving outcomes. Until snapshots exist it reads No snapshot yet. |
The fuller breakdown — decided counts, invalid signals thrown out (e.g. an outcome dated before its case), and the per-kind split — is on the Data sources tab’s match-coverage panel and in the coverage dashboard widgets.
Coverage on a business KPI is your outcome match rate carried forward to the case grain. If it’s low, the fix is almost always upstream on the Data sources tab: read the match coverage, find the correlation type that’s failing to match, and fix it there. An entirely separate reason coverage can be low is that your feed simply only covers some cases (only closed tickets, only one queue) — a real gap that coverage is showing rather than hiding.
Failure impact — which failures are eroding a KPI
Coverage tells you how trustworthy a KPI is. The Failure impact tab tells you what to do about it: pick a KPI, and Neens ranks the failure clusters dragging it down — the recurring failures that are costing you the most containment, or the most money per case — worst first.
For each cluster, Neens computes the KPI two ways over the same window: once across that cluster’s cases, and once across the whole agent (the baseline). The gap between them is the cluster’s erosion — how far its rate sits below the number the rest of your traffic hits. Ranking the clusters by that gap turns “containment is at 87%, and I wish it were higher” into “these three recurring failures are where the missing points went, in this order”.
At a glance
| Where | The Failure impact tab on the Business KPIs page; a chip on each Failure modes card. |
| What it answers | Which failures are eroding this KPI, and by how much each? |
| Defined for | Rate KPIs (containment, and any rate custom measure) and the cost per case KPI. Not a percentile — see below. |
| Key API | GET /kpis/{id}/eroding-clusters |
| Freshness | Recomputed after each analysis run over your recent window (30 days by default). |
| Who can view | Anyone with read access to the agent. |
Reading the ranked list
Every cluster in the ranked list carries three numbers that matter:
- Δ vs baseline (
erosionRate) — how far this cluster’s rate sits below (for containment) or above (for a cost or error rate) the agent baseline. A containment cluster at 58% against an 87% baseline is 29 points of erosion. - Impact share (
impactShare) — this cluster’s fraction of all the KPI’s measured erosion, so the shares of the ranked clusters add toward 100%. This is the number that answers “if I could fix one thing, which?” — and it is deliberately weighted by size, not just by gap. A 29-point miss on 96 decided cases outranks a 40-point miss on 5, because it is responsible for more lost cases. - Decided cases (
clusterDecided) — the denominator the verdict rests on. A big gap over a handful of cases is a smaller, less certain problem than a moderate gap over hundreds.
Impact share is weighted by lost cases, not by the raw gap. Erosion is measured in cases — the count of contained (or cheap) cases the cluster would have delivered at the baseline rate. That is the quantity that adds up across clusters, which is what lets one cluster’s share be compared to another’s. A dramatic percentage gap on a tiny cluster is a small share; a modest gap on a large one can be the biggest share on the page.
Reading the “not enough data yet” list
Clusters with too few decided cases to tell a real regression apart from sampling noise are listed separately, as not enough data yet — never mixed into the ranked list, and never shown as 0 impact.
“Not enough data yet” is not “harmless”. A cluster we could not measure is not a cluster that
erodes nothing — those are different statements, and collapsing the first into the second is exactly
how a real problem hides behind a green 0. So a thin cluster gets its own list with a stated
reason, its erosion and impact share both read — (never 0), and it is never ranked as if it were
fine. It moves into the ranked list on its own once it has enough decided cases.
The reason a cluster is in this list is one of:
| Reason | What it means |
|---|---|
| Not enough decided cases | The cluster has fewer decided cases than the minimum needed to separate signal from noise. Give it time, or fix your coverage so more of its cases are decided. |
| No baseline to compare against | The agent-wide number could not be computed for the window — usually no decided cases at all yet. |
| No measurable cases in the cluster | The cluster carried no decided cases in the window. |
| Cost only partly priced | (cost per case only) Some models in the window have no price, so the cost is a floor — subtracting two floors would produce a number nobody can defend, so Neens declines rather than guess. Fill the gaps under Settings → Model pricing. |
Rate KPIs and the cost KPI — and the one type it can’t attribute
Erosion is defined for the two kinds of KPI where “worse than baseline” has an unambiguous meaning:
- Rate KPIs — containment, and any rate custom measure. Erosion is the proportion gap, always oriented by the KPI’s own direction, so a lower containment cluster and a higher error-rate cluster both rank as erosion.
- Cost per case — erosion is the extra spend a cluster’s cases cost above the baseline cost per case, in dollars. It refuses (lands in the not enough data yet list) whenever the window is only partly priced, for the same reason cost per case itself carries a partial badge.
A percentile KPI has no erosion to attribute. Resolution time (p50/p90) is a percentile, not a
proportion — there is no baseline rate for a cluster to sit below — so the tab says erosion isn’t
defined for this KPI rather than inventing a ranking. The API says the same thing with
attributable: false and two empty lists, so a client can tell “this KPI can’t be attributed” apart
from “nothing is eroding it”.
The chip on a failure mode
You don’t have to open this tab to see the connection. On the Failure modes page, each failure-mode card carries a small chip naming the one KPI it erodes most and its share of that KPI’s total erosion — “Eroding Containment · 34%” — linking the failure you’re triaging straight to the business number it’s costing. A cluster with no measurable erosion carries no chip rather than a 0% one, for the same reason it sits in the not enough data yet list: an empty chip would read as “measured and harmless”.
Measured vs estimated remediation impact
The ranking feeds the remediations queue. When a cluster has a measured erosion of a KPI, the remediation Neens proposes for that cluster takes its impact — the I in its priority score — from that real erosion share, and marks it measured. When there is no measured erosion to lean on, the impact stays a model’s estimate, marked estimated.
A measured priority and a guessed one never render identically. A remediation whose priority is grounded in a KPI’s real erosion by that cluster shows measured; one resting on the model’s own guess shows estimated. Same score, different confidence in it — so you can tell “this is ranked high because it demonstrably costs us containment” apart from “the model thinks this looks important”.
Over the API
GET /kpis/{id}/eroding-clusters returns the same two lists the tab draws: ranked (worst first,
each with its impactShare) and unknown (the thin clusters, with a reason). Read access is enough.
curl -sf "$NEENS_BASE_URL/api/kpis/$KPI_ID/eroding-clusters" \
-H "Authorization: Bearer $NEENS_API_KEY"{
"kpiId": "a68e0cbb059f4b0ba38f409ee2145b19",
"measureKey": "containment_rate",
"unit": "ratio",
"direction": "higher_is_better",
"windowDays": 30,
"attributable": true,
"ranked": [
{
"clusterId": "cl_7f21a3",
"clusterLabel": "Refund policy mis-quoted, customer escalates",
"clusterSessionCount": 118,
"clusterStatus": "active",
"status": "known",
"unknownReason": null,
"clusterCases": 118,
"clusterDecided": 96,
"clusterValue": 0.5833,
"baselineCases": 812,
"baselineDecided": 704,
"baselineValue": 0.8731,
"erosionUnits": 27.82,
"erosionRate": 0.2898,
"ci": [0.485, 0.676],
"distinguishable": true,
"impactShare": 0.624,
"computedAt": "2026-08-19T02:14:07Z"
},
{
"clusterId": "cl_3b90e2",
"clusterLabel": "Order-lookup tool times out mid-conversation",
"clusterSessionCount": 160,
"clusterStatus": "active",
"status": "known",
"unknownReason": null,
"clusterCases": 160,
"clusterDecided": 140,
"clusterValue": 0.7536,
"baselineCases": 812,
"baselineDecided": 704,
"baselineValue": 0.8731,
"erosionUnits": 16.73,
"erosionRate": 0.1195,
"ci": [0.676, 0.820],
"distinguishable": true,
"impactShare": 0.376,
"computedAt": "2026-08-19T02:14:07Z"
}
],
"unknown": [
{
"clusterId": "cl_c14af8",
"clusterLabel": "Non-English greeting not recognised",
"clusterSessionCount": 11,
"clusterStatus": "active",
"status": "unknown",
"unknownReason": "insufficient_decided",
"clusterCases": 11,
"clusterDecided": 7,
"clusterValue": 0.5714,
"baselineCases": 812,
"baselineDecided": 704,
"baselineValue": 0.8731,
"erosionUnits": null,
"erosionRate": null,
"ci": [0.25, 0.84],
"distinguishable": false,
"impactShare": null,
"computedAt": "2026-08-19T02:14:07Z"
}
]
}Reading it:
| Field | What it tells you |
|---|---|
attributable | true for a rate or cost KPI; false for a percentile, where both lists are empty and the tab says erosion isn’t defined for it. |
ranked | The clusters with a defensible erosion, worst first (largest erosionUnits). |
unknown | The thin clusters, listed apart with an unknownReason. Their erosionUnits, erosionRate and impactShare are null — rendered as —, never 0. |
clusterValue · baselineValue | The KPI over this cluster’s cases, and over the whole agent — the two numbers whose gap is the erosion. |
erosionRate | The per-case gap vs baseline, oriented by the KPI’s direction. For containment above, 0.2898 is a 29-point drop. |
erosionUnits | The gap expressed in cases — the “lost” contained (or cheap) cases the cluster is responsible for. This is what the ranking and the shares are computed from. |
impactShare | This cluster’s 0..1 fraction of the KPI’s total measured erosion. The shares of the ranked clusters sum toward 1. null when it couldn’t be formed — never 0. |
distinguishable | Whether the sample can tell this cluster’s rate apart from the baseline. false means the gap is inside sampling noise; a thin cluster reads false and sits in unknown. |
clusterDecided · baselineDecided | The denominators behind each side. A 90% rate over 7 decided cases is not the fact a 90% rate over 700 is. |
computedAt | When this verdict was last recomputed. |
A null renders as ”—”, and it always means “we could not measure this”, never “zero”. A
cluster in the unknown list, a KPI that can’t be attributed, an impact share that couldn’t be
formed — all read as a dash, the same honesty rule the KPI tiles follow.
Fixes that moved this KPI
Erosion tells you what is dragging a KPI down. The other half of the story is what has pushed it back up: the shipped fixes that actually moved this number. Every KPI now lists them, so “containment recovered” arrives with “…and here are the merged fixes that recovered it.”
The list comes from the same Fix outcomes close-out that measures whether a merged fix reduced its failure’s volume. Alongside that volume leg, each close-out measures every active KPI before vs after the deploy; the ones that moved this KPI are gathered here, newest merge first — each with the before→after Neens measured around the deploy and whether the move stood out from sampling noise.
Unmeasurable movers are listed apart, never ranked as 0. A fix whose KPI move could not be
formed honestly — too thin a sample, no before data, or the
definition changed between the windows — is listed
separately in movedByFixesUnknown, with its delta shown as —. It is never folded into the ranked
movers as a zero, the same rule the eroding-clusters
list follows.
Read it over the API
movedByFixes and movedByFixesUnknown ride on GET /kpis/{id}, beside the KPI’s
value. Read access is enough.
curl -sf "$NEENS_BASE_URL/api/kpis/$KPI_ID" \
-H "Authorization: Bearer $NEENS_API_KEY"{
"kpi": {"id": "a68e0cbb059f4b0ba38f409ee2145b19", "measureKey": "containment_rate",
"label": "Self-serve containment", "…": "…"},
"value": 0.8731,
"…": "…",
"movedByFixes": [
{
"closeoutId": "co-9f2a1c",
"remediationId": "rem-4471",
"remediationTitle": "Guardrail over-blocks refunds",
"prUrl": "https://github.com/acme/agent/pull/42",
"status": "improved",
"delta": 0.19, "deltaPct": 30.65, "distinguishable": true,
"before": 0.62, "after": 0.81,
"mergedAt": "2026-07-10T00:00:00Z", "verdictAt": "2026-07-17T02:11:00Z",
"clusterId": "cl_7f21a3"
},
{
"closeoutId": "co-71bd08",
"remediationId": "rem-4390",
"remediationTitle": "Order-lookup tool retries on timeout",
"prUrl": "https://github.com/acme/agent/pull/38",
"status": "flat",
"delta": 0.01, "deltaPct": 1.16, "distinguishable": false,
"before": 0.86, "after": 0.87,
"mergedAt": "2026-07-02T00:00:00Z", "verdictAt": "2026-07-09T02:07:00Z",
"clusterId": "cl_3b90e2"
}
],
"movedByFixesUnknown": [
{
"closeoutId": "co-55c0a2",
"remediationId": "rem-4210",
"remediationTitle": "Escalation-label taxonomy revised",
"prUrl": "https://github.com/acme/agent/pull/31",
"status": "unknown",
"delta": null, "deltaPct": null, "distinguishable": null,
"before": null, "after": null,
"mergedAt": "2026-06-20T00:00:00Z", "verdictAt": "2026-06-27T02:04:00Z",
"clusterId": "cl_c14af8"
}
]
}| Field | What it tells you |
|---|---|
closeoutId · remediationId · remediationTitle · prUrl | Which fix — and a link straight to the merged PR. |
status | improved · flat · regressed · unknown, oriented by the KPI’s direction. Every unknown sits in movedByFixesUnknown, not here. |
before · after | The KPI before the deploy and after it. null (→ —) for an unmeasurable mover. |
delta · deltaPct | The move, in the KPI’s own unit and as a percentage. null when there is no honest delta to draw. |
distinguishable | Whether the move cleared sampling noise. false means it moved, but inside the noise — the list says so rather than celebrate it. null for a cost KPI, which carries no interval. |
mergedAt · verdictAt | When the fix merged, and when its close-out settled. The list is ordered by newest mergedAt first. |
clusterId | The failure cluster the fix targeted — the corner of your traffic whose recovery this move reflects. |
A null here is a dash, and means “we could not measure this move”, never “zero”. A mover in
movedByFixesUnknown, a delta that couldn’t be formed across a definition change — both render as
—, the same honesty rule the KPI tiles and the
eroding-clusters list follow.
The full two-scope breakdown for any of these fixes — the KPI over just the fix’s cluster vs over the whole agent — lives on its Fix outcomes close-out, where the same before→after is reported for both scopes.
Define a KPI
The Definitions tab is where an admin tells Neens what each KPI means for your data. Members can read the effective definition; only an admin can change it.
Open the Definitions tab
It shows the effective definition for each KPI — your declarations merged over sensible platform defaults — with a marker on anything you haven’t set yourself, and a “Not configured yet” banner until you save for the first time.
Pick the signal, then declare what “escalated” means
Choose whether the KPI reads a measured outcome or an inferred classifier label, then pick the kinds and values (or the label set) that mean escalated and contained. The dropdowns are populated from what this agent has actually recorded — the outcome kinds you’ve received and the exact labels the classifier has really emitted, each marked seen in your data or not seen yet. Free-typing a kind no exporter sends is the fastest way to a permanent 0%, so the picker doesn’t offer one.
Save, then check the coverage
Save, then look at the Overview. The coverage badge under each tile tells you how many of the window’s cases your definition actually decided. If it’s low, the fix is on the Data sources tab.
Choosing labels is a real modelling decision, not a checkbox. “Escalated” means the customer had to go somewhere else to get this done — not the agent did something wrong. A hallucination is a serious quality failure, but the agent still handled the case; folding every failure mode into the escalation set quietly redefines containment as “the classifier found nothing”. Watch for a label whose name reads backwards, like “Human escalation not triggered” — that describes a case that was not escalated.
Do it over the API
Everything the tab does is available over REST, so CI or your infrastructure-as-code can set it. Read the effective definition with an agent API key; change it with an admin credential.
export NEENS_BASE_URL="https://your-neens-host" # your Neens origin
export NEENS_API_KEY="nk_live_..." # your agent API key
# Read the effective definition (any read access)
curl -sf "$NEENS_BASE_URL/api/kpi-definitions" -H "Authorization: Bearer $NEENS_API_KEY"
# Discover what you can pick from — the catalogue annotated with what THIS agent has observed
curl -sf "$NEENS_BASE_URL/api/kpi-definitions/options" -H "Authorization: Bearer $NEENS_API_KEY"Declaring a definition is a full replacement, sent by an admin session — omit a KPI to revert it to the platform default. A partial merge is how half a definition produces a number nobody intended, so Neens doesn’t do one.
curl -sf -X PUT "$NEENS_BASE_URL/api/kpi-definitions" \
-H "Authorization: Bearer $SESSION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"definitions": {
"containment": {
"mode": "outcome",
"escalationKind": "escalation", "escalationValue": true,
"containmentKind": "containment", "containmentValue": true
},
"resolution_time": {
"mode": "outcome",
"resolutionKind": "resolution_time",
"resolutionTimeMode": "measured_duration"
}
}
}'resolution_time.resolutionKind names the kind that carries the duration, which is usually not
the same kind as the boolean that says a case was resolved. Point it at a duration kind
(resolution_time, handle_time); pointed at a boolean, the measure reports unavailable and names
the mistake rather than reading true as a one-millisecond case.
A definition is validated at write time, so a malformed one is a 422 on the PUT and never a
silently-wrong number on a dashboard later.
Connect the data behind a measured KPI
The Data sources tab is the measured path’s home. There are two ways to get outcomes in, and both land on this tab:
- Push — your ETL or backend
POSTs outcomes to Neens. Full control over the mapping. - Pull — a scheduled helpdesk connector (Zendesk, Intercom, Salesforce, Jira Service Management) that an admin sets up once.
Either way, the tab shows the match coverage — the single most important number on it — so you can see what fraction of your outcomes actually matched a case before you trust any KPI built on them. Connecting a feed is an admin action; every member can read the coverage. The full model is in Business outcomes.
Inferred KPIs: the ready-made classifiers
Some cases arrive with a fact about what happened — a helpdesk disposition, a CRM outcome. Many don’t. When you have no outcome feed for a question, Neens can still put a number on the board: it reads the transcript with an LLM classifier and labels each case itself. That number is the inferred tier — it wears the purple Inferred badge everywhere it renders, because it is Neens’ opinion about the past, not a record of it.
You don’t have to build a classifier to use this path. Neens ships three ready-to-use ones, each labelling a case on a single axis:
| Classifier | The label it writes on each case | What it powers |
|---|---|---|
| Escalation | escalated_to_human · handoff_requested · contained | An inferred containment rate — the share of decided cases it labels contained. |
| Resolution | resolved · unresolved · unclear, judged from the closing turns | An inferred resolution rate. |
| Sentiment | frustrated · neutral · satisfied | A directional read on how a case felt — back it with a custom measure. |
An inferred number is a starting point, not a destination. It exists so a brand-new agent isn’t stuck on “no data” while it wires up a real feed — a directional read you can act on today, honestly badged so nobody mistakes it for the customer’s own word. As soon as the measured signal lands, point the KPI at that; the inferred one was scaffolding, and Neens tells you how far it agreed with reality so you know when it’s safe to lean on.
Turn a classifier on
Enable the classifier
The three classifiers are opt-in. Turn one on from your agent’s classifiers area, alongside your judges and scorers. Until you do, it writes nothing and costs nothing — an inferred containment tile has no labels to read and stays blank.
Let it label your cases
Once it’s on, Neens starts reading conversations and writing that classifier’s label onto each case it samples. Because it uses an LLM, it runs on your agent’s configured LLM connection (Settings → Connections); an agent with no connection can’t label, so its inferred tiles stay blank until one is set.
Point a KPI at the labels
On the Definitions tab, set the KPI’s signal to Inferred classifier label and
choose the labels that mean escalated — for the Escalation classifier, escalated_to_human and
handoff_requested, leaving contained as the settled side. The picker only offers labels the
classifier has actually emitted, each marked seen in your data or not seen yet, so you can’t
typo your way to a permanent 0%. Save, and an Inferred containment tile appears.
An inferred KPI can never produce a resolution time. A classifier label is a judgement about a transcript; it records no moment at which the customer’s problem was settled. So the Resolution classifier can back a resolution rate, but resolution time stays unavailable with that reason on any classifier definition — Neens won’t fall back to the trace’s own duration, which measures how long the agent ran, not how long the case took.
The classifiers sample your whole population
Failure clustering looks only at your failing sessions — that’s the whole point of it. The inferred classifiers do the opposite. They sample across your entire population: a share of your traffic each day, spread across your agents and your outcomes, so the resulting rate reflects the fleet, not just the cases that went wrong. A containment rate built from failures alone would describe nothing; one built from a representative slice of everything is a real estimate.
Reading a slice rather than every case is deliberate — scoring every conversation with an LLM would be costly and unnecessary. A well-spread daily sample estimates a fleet-wide rate without touching every conversation, which is exactly why the coverage on an inferred tile is expected to be a fraction, and says so.
Read the coverage on an inferred tile: “sampled” is not “missing”
Every KPI tile carries a coverage figure. On a measured KPI, a low
coverage is usually a problem — a feed that isn’t matching, a queue you forgot to send. On an
inferred KPI it is the design working: Neens scored a representative sample, not the whole
population, so the tile labels its coverage sampled rather than leaving you to read 12% as a
data gap.
So on an inferred containment tile you’ll see something like sampled · 12%/day across agents and outcomes. Read that as “Neens measured roughly 12% of this window’s cases each day, balanced across your agents and outcomes” — a healthy sample, not 88% missing data. Hover the coverage for the raw measured X of Y cases so far.
A 12% on a measured tile and a 12% on an inferred tile mean opposite things. On the measured
tile, 88% of your cases had no outcome — go fix the feed. On the inferred tile, Neens deliberately
sampled 12% to keep the read cheap and representative — nothing is wrong. The sampled label is how
you tell the two apart at a glance, so you never chase a coverage gap that was the plan all along.
Calibration: does the inference agree with reality?
An inferred number is only worth trusting if it agrees with what really happened. An LLM classifier can be systematically wrong — too eager to call a case escalated, too generous about resolution — and a containment rate printed to two decimals hides exactly that uncertainty.
So whenever an inferred KPI also has a measured counterpart — inferred containment on the cases where your helpdesk also recorded an escalation, say — Neens computes a calibration: how well the classifier’s labels agree with the measured outcome on the cases where you have both. It appears on the tile as a plain sentence:
inferred containment agrees with measured at κ=0.71 over 312 cases
κ is Cohen’s kappa — chance-corrected agreement. Read it on a simple scale: 1.0 is perfect agreement, ~0 is no better than a coin flip, and a negative κ means the classifier disagrees with reality more often than random guessing would. So κ=0.71 over 312 cases is a strong signal that this classifier is tracking your real escalations; you can lean on the inferred number where the measured one runs out.
When there aren’t enough matched cases yet, Neens says so — it never prints a number it can’t stand behind. A κ computed from a dozen labels is a coin flip wearing a decimal point, so below a minimum count of matched pairs the calibration reads “Not enough measured cases to calibrate yet” instead of a figure. A KPI that is not inferred (an outcome-backed one) has no classifier to check, so it shows no calibration at all — there’s nothing to calibrate against itself.
Reading a low κ. A κ near 0 (or below it) is telling you the inference has drifted from reality — the classifier and your system of record are describing different worlds. When that happens, trust the measured KPI over the inferred one, and either retune what you count as escalated on the Definitions tab or, better, move the KPI onto the measured feed entirely. Calibration is the dial that tells you when the scaffolding has done its job and can come down.
The full calibration block — κ plus the classifier’s precision and recall against the human labels,
over a trailing window — is available over the API at
GET /kpis/{id}/calibration.
End-to-end: Acme turns on Escalation
Acme’s support agent has traces but no helpdesk feed wired up yet, so its containment tile is blank.
Enable the Escalation classifier
An admin turns on the Escalation classifier. Neens begins sampling Acme’s conversations and
labelling each escalated_to_human, handoff_requested or contained.
Declare inferred containment
On Definitions, Acme sets containment to Inferred classifier label and picks
escalated_to_human and handoff_requested as the escalated labels. An Inferred containment tile
appears reading 78%, with coverage shown as sampled · 14%/day — the sampler working as
intended, not a missing 86%.
Wire up the helpdesk, and watch the calibration
A week later Acme connects its helpdesk outcome feed. Now some cases carry both the classifier’s label and a recorded escalation, so the tile grows a calibration line: inferred containment agrees with measured at κ=0.71 over 312 cases. κ=0.71 is strong, so Acme trusts the inferred 78% for the corners its feed doesn’t yet cover — and knows exactly when to switch the KPI over to the measured signal.
Add a measure of your own
Three named KPIs prove the idea; they don’t describe your business, because “handled it” means something different for support, sales, IT and collections. The Custom measures tab lets an admin declare a measure in your own vocabulary — your label, your source, your denominator — that then behaves exactly like one Neens ships: it charts on a dashboard, backs an alert rule, and carries its own provenance and coverage. A custom measure’s source decides its badge, the same four tiers as above. See Custom measures for the walkthrough.
Starting from nothing
A brand-new agent isn’t stuck. The Overview offers three real ways to a first number, in increasing order of trust:
- Cost per case, today. It’s derived from your traces and prices — no feed, no definition. As soon as you have priced traffic it has a value, which you read on Cost optimization (the empty state links you straight there).
- Connect a feed, for a measured number. Wire up an outcome feed on the Data sources tab and declare containment against it — the strongest evidence there is.
- Attach a classifier, for a directional number. No feed yet? Turn on a ready-made classifier — escalation, resolution or sentiment — or point containment at your issue-classifier labels, for an inferred rate while you build the real thing. Neens calibrates it against your measured feed once both exist, so you know when to trust it.
Promote a measure to a KPI
A definition says what a number means. A KPI says what you promised about it: the figure you committed to, which direction is good, who owns it, whether the commitment is live, and how often somebody is supposed to look at it. Those are different facts with different lifetimes — a definition changes when your data changes, a commitment changes when your goals do — so Neens keeps them apart.
| Thing | The question it answers | Scope |
|---|---|---|
| KPI definition | What does “escalated” mean in our data? | Agent · the three business measures |
| Custom measure | What number do we care about that isn’t in the catalogue? | Company |
| KPI | What did we commit to on this number, and who owns that promise? | Agent |
A KPI is deliberately thin. It references a measure key and carries only the promise — it never redefines a measure, never recomputes anything, and holds no number of its own. Every value a KPI displays is resolved through the same path a dashboard widget uses, which is what guarantees that a KPI and a widget pointed at the same measure can never disagree: not about the value, not about the provenance, not about the coverage, and not about the blank.
Anything in the catalogue can be promoted — the three business measures on this page, an
operational measure like error rate, or one of your own
custom measures (custom:<slug>).
The fields, and what each one is for
| Field | Required | What it’s for |
|---|---|---|
measureKey | ✅ | Which measure this is a commitment about. Immutable — see Changing one. |
label | ✅ | The name your organisation uses for it, which is rarely the catalogue’s. Max 120 characters. |
description | — | Why this is a commitment and what a reader should do about it. Max 500 characters. |
direction | defaulted | higher_is_better or lower_is_better. This is the field that decides whether a value above the target is a success or a failure — nothing else does. |
target | — | The committed figure, in the measure’s own unit. null is a real state, not zero. |
owner | — | Free text: a person, a team, a rota. Max 200 characters. Neens does not resolve it to an account — the point is that a name appears next to a promise. |
status | draft | draft (authored, not committed) → active (live) → archived (retired). |
priority | 100 | Display rank, 1–999, lower sorts first. The default is mid-range so a new KPI lands after the ones somebody bothered to rank, without renumbering the list. |
reviewCadence | none | none · weekly · monthly · quarterly. Declarative: it records how often this is supposed to be looked at. none is explicit rather than a null, so “deliberately unscheduled” and “nobody decided” stay distinguishable. |
Direction is proposed, never inferred. Neens pre-fills it from the measure’s semantics —
containment is higher_is_better, cost per case and resolution time are lower_is_better, a
custom measure reuses the direction its author already declared — and you can always override it.
What is stored is what every later read uses; nothing looks at the data and guesses which way is
good at render time.
A ratio target is a fraction, not a percentage. containment_rate has unit ratio, so an 85%
commitment is 0.85. A target of 85 is rejected with a 422 naming the unit rather than stored —
otherwise the KPI would read missed forever while the agent was doing fine. Targets on count,
ratio, ms, seconds and usd measures must also be >= 0.
Find out what you can promote
GET /kpis/options returns the measures this agent may promote, each annotated with two things
you want to know before you commit: whether the agent can actually get a number out of it
today (ready), and whether it is already promoted.
curl -sf "$NEENS_BASE_URL/api/kpis/options" \
-H "Authorization: Bearer $NEENS_API_KEY"{
"projectId": "proj_x",
"measures": [
{
"key": "containment_rate",
"label": "Containment rate",
"unit": "ratio",
"agg": "rate",
"category": "business",
"grain": "case",
"isCustomMeasure": false,
"provenance": null,
"defaultDirection": "higher_is_better",
"promoted": false,
"promotedKpiId": null,
"ready": true,
"unavailable": null
},
{
"key": "cost_per_case",
"label": "Cost per case",
"unit": "usd",
"agg": "ratio",
"category": "business",
"grain": "case",
"isCustomMeasure": false,
"provenance": null,
"defaultDirection": "lower_is_better",
"promoted": true,
"promotedKpiId": "3d81f0a742be4c1e9a05d6b3f81c47ab",
"ready": true,
"unavailable": null
},
{
"key": "custom:deflection-rate",
"label": "Deflection rate",
"unit": "ratio",
"agg": "rate",
"category": "business",
"grain": "case",
"isCustomMeasure": true,
"provenance": "measured",
"defaultDirection": "higher_is_better",
"promoted": false,
"promotedKpiId": null,
"ready": false,
"unavailable": {"reason": "no_matching_outcomes"}
}
],
"directions": ["higher_is_better", "lower_is_better"],
"statuses": ["draft", "active", "archived"],
"reviewCadences": ["none", "weekly", "monthly", "quarterly"],
"max": 24,
"remaining": 21
}The list is ordered so the top of it is something you can promote right now: ready measures first, then ones already promoted, then alphabetically.
| Annotation | What it means |
|---|---|
ready | Whether this agent can get a number out of the measure today. |
unavailable | null when ready, otherwise {"reason": "<code>"} — definition_not_configured (the measure’s meaning hasn’t been declared yet), no_matching_outcomes or no_matching_scores (the signal the measure reads has never arrived in this agent). |
promoted / promotedKpiId | Whether a live KPI already exists on this measure, and which one — what keeps you from creating a second commitment on the same number. |
defaultDirection | The proposal you can accept or override. |
provenance | For a custom measure, the badge it will carry (Measured / Inferred). null for a platform measure, which decides its own. |
max / remaining | Your agent’s cap and what’s left of it. null means no cap. Read them rather than assuming a figure. |
ready: false does not stop you promoting the measure — committing to a number you are about
to start collecting is perfectly reasonable. It tells you the tile will read blank today, so the
blank is expected rather than alarming, and it names the page that fixes it.
Create one
Pick the measure
From GET /kpis/options above. Take defaultDirection with it: you only need to send direction
when you disagree with the proposal.
POST the commitment
Only measureKey and label are required. Everything else has a default, and every default is the
conservative one — draft, no target, priority 100, no review cadence.
curl -sf -X POST "$NEENS_BASE_URL/api/kpis" \
-H "Authorization: Bearer $SESSION_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"measureKey": "containment_rate",
"label": "Self-serve containment",
"description": "Share of support cases the assistant closes without a human. Board metric for FY27.",
"direction": "higher_is_better",
"target": 0.85,
"owner": "Support Platform (@rota-support)",
"status": "active",
"priority": 10,
"reviewCadence": "monthly"
}'{
"kpi": {
"id": "a68e0cbb059f4b0ba38f409ee2145b19",
"projectId": "proj_x",
"measureKey": "containment_rate",
"label": "Self-serve containment",
"description": "Share of support cases the assistant closes without a human. Board metric for FY27.",
"direction": "higher_is_better",
"target": 0.85,
"owner": "Support Platform (@rota-support)",
"status": "active",
"priority": 10,
"reviewCadence": "monthly",
"isCustomMeasure": false,
"createdAt": "2026-08-14T10:22:41+00:00",
"createdBy": "[email protected]",
"updatedAt": "2026-08-14T10:22:41+00:00",
"updatedBy": "[email protected]",
"archivedAt": null
}
}The response is a 201. A KPI created without status stays a draft: it is authored, it is not
yet a promise, and it does not appear on the summary.
Commit to a figure when you’re ready
A KPI with "target": null is a legitimate, common state — we watch this number, we have not
committed to a figure. Promote first, argue about the number later:
KPI_ID=a68e0cbb059f4b0ba38f409ee2145b19
curl -sf -X PATCH "$NEENS_BASE_URL/api/kpis/$KPI_ID" \
-H "Authorization: Bearer $SESSION_TOKEN" \
-H "Content-Type: application/json" \
-d '{"target": 0.88, "status": "active", "reviewCadence": "quarterly"}'Read the current value
GET /kpis/{id} returns the KPI and the number, resolved over the range you ask for
(?range=30d).
curl -sf "$NEENS_BASE_URL/api/kpis/$KPI_ID?range=30d" \
-H "Authorization: Bearer $NEENS_API_KEY"{
"kpi": {"id": "a68e0cbb059f4b0ba38f409ee2145b19", "measureKey": "containment_rate",
"label": "Self-serve containment", "direction": "higher_is_better",
"target": 0.85, "status": "active", "…": "…"},
"value": 0.8731,
"unit": "ratio",
"provenance": "measured",
"coverage": {"cases": 812, "decided": 704, "rate": 0.867, "invalid": 0, "asOf": "2026-08-14T11:04:18Z"},
"definitionVersion": "02f3318341c0df32",
"pricing": null,
"unavailable": null,
"targetStatus": "met",
"trend": {"direction": "improved", "current": 0.8731, "previous": 0.8104,
"delta": 0.0627, "pctDelta": 7.73692, "fromDay": "2026-08-06",
"toDay": "2026-08-13", "lookbackDays": 7, "unavailable": null},
"trendUnavailable": null,
"range": "30d",
"asOf": "2026-08-14T11:04:18Z"
}Everything below value is there for the same reason the rest of this page exists: a number without
its evidence is not a number you can act on.
| Field | What it tells you |
|---|---|
value | The measure’s current value over range. null when it could not be computed — never 0. |
unit | The measure’s unit (ratio, ms, usd, count, …) — the unit target is in, too. |
provenance | Measured or Inferred, exactly as on a widget. Promoting a measure does not upgrade its evidence. |
coverage | The block documented under Coverage: how many cases the window held, and how many the definition could decide. null when the measure produced no coverage to report. |
definitionVersion | The fingerprint of the KPI definition this number was computed under. If it changed since you last looked, the meaning of “escalated” changed — and so did the number, for reasons that have nothing to do with the agent. |
pricing | null unless the measure is priced. On a cost measure it carries the pricing detail for the window — see the callout below. |
unavailable | null when the value came out. Otherwise {"reason": …}, naming why the value could not be computed. |
targetStatus | met · missed · unknown. See below. |
trend | Direction of travel, read off recorded history — improved, flat or regressed, plus the two days and values behind it. null when there is no drawable line. |
trendUnavailable | Why there is no trend: no_snapshots, insufficient_history or definition_changed. Separate from unavailable, because “we have no history” and “we have no value” are different facts and a tile has to be able to show one without the other. |
range · asOf | The window, and when this was computed. Non-negotiable on a business number — see late arrival. |
movedByFixes · movedByFixesUnknown | The shipped fixes that moved this KPI — merged remediations whose close-out registered a move on this number, with the unmeasurable ones listed apart. |
A non-null pricing block means the figure is a floor, not a value. When some of the models in
the window have no price, their tokens are counted and their dollars are not, so a cost_per_case
KPI reads lower than the truth. The block is what lets the number be badged partial instead of
quietly under-reporting spend — and a target judged met against an under-reported cost is exactly
the wrong answer to get silently. Fill the gaps under Settings → Model pricing; see
Cost & model pricing.
Met, missed, and the answer most dashboards get wrong
targetStatus compares value against target in the KPI’s declared direction, inclusively —
hitting the target exactly is meeting it, in both directions.
direction | met when | missed when |
|---|---|---|
higher_is_better | value >= target | value < target |
lower_is_better | value <= target | value > target |
No value means unknown. It never means missed. A KPI whose window produced no decided cases
has not been missed — nobody knows whether it was. Rendering a red missed tile for absent data is
how a dashboard manufactures a crisis for an agent that simply hasn’t sent outcomes yet, and it is
the same mistake as counting an unmeasured case as contained. When value is null, targetStatus
is unknown and unavailable.reason says why in a sentence you can act on.
A null target is not a target of zero. “We watch this number but haven’t committed to a
figure” is a real state — it is what every KPI looks like on day one. It also reads unknown: there
is nothing to have met or missed. A lower_is_better KPI with a null target is not silently
holding you to 0.
So a KPI with no data this window looks like this — and says why, in the same breath:
{
"kpi": {"id": "a68e0cbb059f4b0ba38f409ee2145b19", "label": "Containment rate",
"direction": "higher_is_better", "target": 0.72, "status": "active", "…": "…"},
"value": null,
"unit": "ratio",
"provenance": "measured",
"coverage": null,
"definitionVersion": "02f3318341c0df32",
"pricing": null,
"unavailable": {"reason": "Business outcomes are not available in this workspace yet, so containment cannot be measured."},
"targetStatus": "unknown",
"trend": null,
"trendUnavailable": {"reason": "no_snapshots"},
"range": "30d",
"asOf": "2026-08-19T04:49:16.109253+00:00"
}Note the shape: unavailable is flat, and its reason is usually a readable sentence rather
than a code — it is written to be shown to the person looking at the tile, so treat it as prose and
don’t switch on it. Three values are stable codes worth branching on: measure_removed (the measure behind the KPI was deleted), measure_redefined (a measure with this KPI’s key exists again, but it is not the one that was promoted — see below) and no_data_in_window (the measure resolved, but the window you asked for selected nothing). Everything else is prose that can be reworded, so match on those three and render the rest verbatim.
Three more honesty rules ride on that block:
trendisnullwhenever there is no drawable line, andtrendUnavailable.reasonsays which of the three reasons it is — hereno_snapshots, because an agent with no outcomes has no recorded days either. Neens does not draw a sparkline it cannot back with recorded history, and it never interpolates one across a definition change. A missing trend never suppresses the value: the two live in separate fields precisely so a KPI can show today’s number while saying it cannot yet show the direction of travel.- If the custom measure behind a KPI is deleted,
unavailable.reasonis the literalmeasure_removedrather than a number. The commitment survives the measure — you can see that the promise is now pointing at nothing, which is a fixable situation, instead of reading a wrong figure that looks fine. See before you delete a shared measure. coverageandasOfcome with every value, always. Outcomes arrive late by design, so re-reading the same window tomorrow can legitimately show higher coverage and a different value. Nothing changed retroactively; more evidence arrived about the same cases.
Every active KPI at once
GET /kpis/summary is the scorecard: every active KPI with its value, in display order (live
before retired, then priority ascending, then label — deterministic, so two identical reads never
render in two different orders).
curl -sf "$NEENS_BASE_URL/api/kpis/summary?range=30d" \
-H "Authorization: Bearer $NEENS_API_KEY"{
"projectId": "proj_x",
"range": "30d",
"version": "a17be0c94d2f5561",
"asOf": "2026-08-14T11:04:18Z",
"count": 2,
"truncated": false,
"kpis": [
{
"kpi": {"id": "a68e0cbb059f4b0ba38f409ee2145b19", "measureKey": "containment_rate",
"label": "Self-serve containment", "direction": "higher_is_better",
"target": 0.85, "priority": 10, "…": "…"},
"value": 0.8731, "unit": "ratio", "provenance": "measured",
"coverage": {"cases": 812, "decided": 704, "rate": 0.867, "invalid": 0, "asOf": "2026-08-14T11:04:18Z"},
"definitionVersion": "02f3318341c0df32", "pricing": null, "unavailable": null,
"targetStatus": "met",
"trend": {"direction": "improved", "current": 0.8731, "previous": 0.8104,
"delta": 0.0627, "pctDelta": 7.73692, "fromDay": "2026-08-06",
"toDay": "2026-08-13", "lookbackDays": 7, "unavailable": null},
"trendUnavailable": null,
"range": "30d", "asOf": "2026-08-14T11:04:18Z"
},
{
"kpi": {"id": "3d81f0a742be4c1e9a05d6b3f81c47ab", "measureKey": "cost_per_case",
"label": "Cost to serve", "direction": "lower_is_better",
"target": 0.42, "priority": 20, "…": "…"},
"value": 0.5108, "unit": "usd", "provenance": "measured",
"coverage": {"cases": 812, "decided": 812, "rate": 1.0, "invalid": 0, "asOf": "2026-08-14T11:04:18Z"},
"definitionVersion": null, "pricing": {"…": "the pricing detail behind the dollars"},
"unavailable": null,
"targetStatus": "missed",
"trend": null, "trendUnavailable": {"reason": "insufficient_history"},
"range": "30d", "asOf": "2026-08-14T11:04:18Z"
}
]
}Each entry is exactly the GET /kpis/{id} block above — same fields, same rules — so a scorecard
row and a detail view can’t tell you different things about the same commitment.
version is a content fingerprint of the KPIs themselves. It changes when a commitment changes —
a new target, a flipped direction, an archive — and not when somebody re-saves a KPI without
changing anything, so a cached scorecard is not invalidated by a no-op edit.
count is how many rows came back, and truncated is the one you should never see: it is true
when the per-agent cap was lowered underneath data that
already existed, so the tail of the list was dropped. It is reported rather than hidden, because a
silently shortened scorecard reads as “these are all your KPIs” — which would be a lie about a
promise somebody made. Archive down to the cap and it goes back to false.
Drafts and archived KPIs are excluded here. To see them, list with an explicit status:
curl -sf "$NEENS_BASE_URL/api/kpis?status=all" \
-H "Authorization: Bearer $NEENS_API_KEY"{"projectId": "proj_x", "kpis": ["…"], "version": "a17be0c94d2f5561", "count": 3, "max": 24}GET /kpis without a status returns everything except archived. status=active, draft,
archived and all narrow or widen it.
Changing one
PATCH /kpis/{id} is a genuine partial update: only the keys you send are touched, so
{"target": 0.9} cannot blank the description you didn’t mention. An explicit null is
meaningful — it withdraws a committed target, or clears a free-text field — so it is the absence
of a key, not its null value, that means “leave this alone”.
measureKey cannot be changed. Repointing a KPI at a different measure is a 422, not a silent
success. Every review note, screenshot and conversation that cites “Self-serve containment” would
silently start describing a different number, retroactively. Promote the other measure as its own
KPI and archive this one — that leaves an honest record of what was promised and when it changed.
{"detail": "measure_key cannot be changed on an existing KPI. History that already cites this KPI would silently change meaning. Create a new KPI instead."}Retiring one
curl -sf -X POST "$NEENS_BASE_URL/api/kpis/3d81f0a742be4c1e9a05d6b3f81c47ab/archive" \
-H "Authorization: Bearer $SESSION_TOKEN"Archiving retires the commitment without deleting it. An archived KPI keeps its history — the
history of what a team promised is most of the point of having promised it — is excluded from
GET /kpis/summary and from the default GET /kpis listing, stops counting against the
one-per-measure rule, and stamps archivedAt. Nothing about the underlying measure changes: the
widget on your dashboard keeps working, because a KPI never owned that number in the first place.
One live KPI per measure, and the cap
Two limits, both enforced loudly:
-
One live KPI per measure per agent. Two simultaneous commitments to
containment_ratemeans two targets, and the honest answer to “did we hit it?” becomes “which one?”. Creating a duplicate is a409naming the KPI that already exists, so you can go and edit it:{"detail": "This project already has a KPI on 'containment_rate' ('Self-serve containment', id a68e0cbb059f4b0ba38f409ee2145b19). Edit it, or archive it first."}Archived KPIs are exempt. You can re-commit to a measure you retired last quarter — the old commitment stays in the record and the new one starts clean.
-
A per-agent cap on KPIs. Reaching it is a
409naming the limit, never a silent drop:{"detail": "This project already has 12 KPIs, the maximum is 12. Archive one you no longer commit to, then try again."}Read the live figures from
max/remainingonGET /kpis/options(ormaxonGET /kpis) rather than hard-coding one. The cap exists because a scorecard with sixty rows is not a scorecard — it is a table nobody reads, and every entry on it stops meaning “we promised this”.
KPI endpoints and vocabularies, in full
| Endpoint | What it does |
|---|---|
GET /kpis | List. ?status=active|draft|archived|all; the default is everything except archived. Returns {projectId, kpis, version, count, max}. |
GET /kpis/options | The promotable measures, each with ready / unavailable and promoted / promotedKpiId, plus the vocabularies and max/remaining. |
POST /kpis | Create. 201 {kpi}. 409 on a duplicate measure or the cap; 422 on a bad field. |
GET /kpis/{id} | The KPI plus its current value and the fixes that moved it (movedByFixes / movedByFixesUnknown). ?range=30d. |
PATCH /kpis/{id} | Partial update. 422 on measureKey or an unknown field. |
POST /kpis/{id}/archive | Retire the commitment; keeps the history. |
GET /kpis/summary | Every active KPI with its value, in display order. ?range=30d. Returns {projectId, range, version, asOf, count, truncated, kpis}, where each entry is the GET /kpis/{id} block. |
GET /kpis/{id}/eroding-clusters | The failure clusters eroding this KPI: ranked (worst first, each with impactShare) and unknown (thin clusters, with a reason), plus attributable — false for a percentile KPI. |
GET /kpis/{id}/history | The recorded days for one KPI, oldest first, plus trend, breaks and definitionChangedAt. ?days=30, clamped; 422 on days=0. |
POST /kpis/{id}/snapshot | Record one KPI’s recent days now. ?days= to redo a specific stretch. Idempotent. |
POST /kpis/snapshot | The same for every active KPI in the agent. |
Every route is also reachable at its bare path (/kpis) as well as under /api.
| Vocabulary | Values |
|---|---|
direction | higher_is_better · lower_is_better |
status | draft · active · archived |
reviewCadence | none · weekly · monthly · quarterly |
targetStatus | met · missed · unknown |
priority | Integer 1–999, lower sorts first. Default 100. |
unavailable.reason (value) | Free text meant for a reader — not a closed enum. measure_removed, measure_redefined and no_data_in_window are stable codes; anything else is prose to render as-is. |
unavailable.reason (options) | definition_not_configured · no_matching_outcomes · no_matching_scores |
trendUnavailable.reason | no_snapshots · insufficient_history · definition_changed |
trend.direction | improved · flat · regressed · unknown |
History and trends
A KPI’s value is resolved live, over whatever window you ask for. That answers where are we now? and it can never answer are we getting better? — a live query knows what last Tuesday looks like today, not what it looked like on the Wednesday after. Outcomes land late, prices change, definitions get sharpened; the number you’d read for a past day today is not the number that day actually had.
So Neens records each active KPI once per completed day, and every trend is read off those
recorded days rather than re-derived. You don’t have to start it: once a KPI is active, its
history accumulates on its own, and it keeps accumulating whether or not anyone opens a chart. The
two snapshot routes below exist for when you want a day recomputed right now rather than
whenever the next pass comes round.
| Route | What it does |
|---|---|
GET /kpis/{id}/history | The recorded days for one KPI, oldest first, plus the trend and any series breaks. |
POST /kpis/{id}/snapshot | Record (or re-record) one KPI’s recent days now. |
POST /kpis/snapshot | The same, for every active KPI in the agent. |
GET /kpis/{id} and GET /kpis/summary grow a real trend from the same recorded days — the
trendUnavailable: {"reason": "no_snapshots"} placeholder now only shows up when there genuinely
is no history yet.
What a trend is, and how to read it
A trend compares the newest recorded day that has a value against a day at least 7 days older. A week, not a day: a business number with a weekly shape — support volume, containment, cost — gets compared against the same weekday, so what you read is a change in the business rather than the fact that Sunday is quiet.
| Field | What it tells you |
|---|---|
direction | improved · flat · regressed · unknown. Read through the KPI’s own direction: a falling cost_per_case is improved; a falling containment_rate is regressed. |
current / previous | The two values compared — both real recorded values, never interpolated. |
delta | current − previous, in the measure’s own unit. Signed arithmetically, so you can render the arrow yourself; direction is the one that already knows which way is good. |
pctDelta | The same movement as a percentage, or null when previous is 0. A percentage change from nothing is not a number, and a tile reading Infinity% is worse than a blank. |
fromDay / toDay | The two days compared. Both are complete UTC days. |
lookbackDays | How far apart the two days had to be — 7. |
unavailable | null on a real trend. {"reason": …} whenever direction is unknown. |
flat is a genuine fourth answer, not a rounding artefact: a movement smaller than floating-point
noise is reported as unchanged rather than as a microscopic improvement that flips sign every
time the page reloads.
The three ways a trend reads unknown
unknown is a first-class answer, and it is deliberately three different answers — because whether
waiting fixes it is exactly what you need to know:
unavailable.reason | What happened | What to do |
|---|---|---|
no_snapshots | Nothing has been recorded for this KPI yet — it was promoted minutes ago, or it isn’t active. | Wait, or POST /kpis/{id}/snapshot. |
insufficient_history | There are recorded days, but not two comparable ones a week apart with values in them. | Wait. A KPI promoted on Monday has no trend until the following Monday. |
definition_changed | There is older history, and it measured something else. See series breaks. | Nothing to fix. The trend restarts from the change. |
On GET /kpis/{id} and GET /kpis/summary an unknown trend is reported as trend: null plus
trendUnavailable: {"reason": …} — there is no line to draw, so no line is drawn, and the value
beside it is unaffected. GET /kpis/{id}/history always returns the full trend block, unknown
and all, because a client charting the series needs to know why it isn’t drawing an arrow.
Only completed days are recorded
The newest day that can appear in a history is yesterday, UTC. Today is never a point on the line.
That is not caution for its own sake. A day still filling up is a partial window recorded as if it were a whole one: containment drops off a cliff at 09:00 and heals by itself overnight, cost per case reads a third of its real figure, and somebody escalates. Every day on the line covers the same 24 hours as every other day on it, which is the only way two points on one axis are comparable at all.
The current value on GET /kpis/{id} is unaffected — that is still a live read over the range
you ask for, today included. The history and the tile answer two different questions on purpose.
A day with no number is recorded as a day with no number
A recorded day whose measure produced nothing is written down anyway, with value: null and an
unavailableReason. It is not skipped, and it is not a zero.
That matters because “we looked on the 12th and there was nothing” is history, and it is a different statement from “we never looked” and from “the measure was deleted”. Collapsing all three into a gap in the series throws away the only evidence that distinguishes them.
Two consequences you’ll see directly:
- Every point carries its own
targetStatus, and a point with no value readsunknown— nevermissed. A day with no decided cases has not broken the commitment; nobody knows whether it did. This is the same rule the live tile follows, and it is why a history chart never paints absent days red. - A trend steps over an empty day rather than stopping at it. The comparison uses the newest day that has a value, so a quiet Sunday in the middle of the window does not erase the trend of the days around it.
Recent days get corrected as outcomes arrive
A recorded day is not written once and frozen. Outcomes arrive late — a ticket closed on Thursday decides a case that started on Monday — so what Monday’s containment was keeps changing for a few days after Monday ends. Neens therefore re-checks the most recent days on every pass and rewrites them when the answer changed.
Three fields on each point tell you exactly what happened to it:
| Field | Meaning |
|---|---|
computedAt | When this day was first recorded. It never moves again. |
asOf | When it was last re-checked. This moves on every pass, whether or not anything changed. |
revisions | How many times the recorded fact actually changed — the value, the reason it was absent, or the definition behind it. Re-reading a day forty times is not forty revisions, and a counter that said so would be worth nothing. |
So revisions: 0 with a recent asOf means we keep checking, and the number keeps coming out the
same — which is a stronger statement than a number nobody re-examined. Days far enough in the past
stop being re-checked at all; their asOf stops moving, and that is the number settling, not the
recording stopping.
Re-recording a day rewrites it — it never appends. Each (agent, KPI, day) is one row, so
calling POST /kpis/snapshot five times in a row leaves the history exactly as it was after the
first call, with a newer asOf and the same revisions. Re-running a window is always safe.
A definition change breaks the series
A KPI is a commitment to a measure, and what a measure means is something your agent
declares. Change the declaration — repoint escalationKind, edit the escalation
labels, change a custom measure’s rule — and the days either side of that
change are two different quantities. Subtracting one from the other produces a movement nobody made.
Neens records the fingerprint of the definition behind every day, and reports the change instead of averaging across it:
breakslists each change: the first day under the new definition, and the fingerprints it movedfromandto.definitionChangedAtis the most recent of those — the day a chart should annotate.- Each point carries
comparableWithPrevious, so a client can break the line rather than draw through it. - A trend that would span a break resolves
unknownwithreason: definition_changed, rather than a number.
This is the over-time twin of a rule you may already have met: a widget scoped across agents that define a KPI differently reports unavailable rather than an average. Same argument, one axis over.
A break is not an error, and it does not delete anything. The older days stay readable and stay
correct under the definition they were measured with. What you lose is the right to subtract
across the boundary — which you never had. When a change was a correction rather than a
redefinition, re-record the affected days (POST /kpis/{id}/snapshot?days=…) so the whole window
is measured the new way and the series becomes continuous again.
Read a KPI’s history
curl -sf "$NEENS_BASE_URL/api/kpis/$KPI_ID/history?days=30" \
-H "Authorization: Bearer $NEENS_API_KEY"{
"kpiId": "a68e0cbb059f4b0ba38f409ee2145b19",
"measureKey": "containment_rate",
"unit": "ratio",
"direction": "higher_is_better",
"target": 0.72,
"days": 30,
"points": [
"…",
{
"id": "09ed0073f8232afc9ba68bc00669fccc74484dea",
"projectId": "proj_x",
"kpiId": "a68e0cbb059f4b0ba38f409ee2145b19",
"measureKey": "containment_rate",
"day": "2026-08-11",
"value": 0.3333333333333333,
"unit": "ratio",
"provenance": "measured",
"coverage": {"cases": 3, "decided": 3, "rate": 1.0, "invalid": 0, "asOf": "2026-08-18T02:14:07.540138+00:00"},
"unavailableReason": null,
"unavailable": null,
"definitionFingerprint": "a8bb2e3b1a48b2d8",
"measureVersions": {"kpiDefs": "1:2026-08-02T09:12:44+00:00", "customMeasures": "0:", "priceTable": "3:2026-07-30T00:00:00+00:00"},
"asOf": "2026-08-18T02:14:07.540138+00:00",
"computedAt": "2026-08-12T02:11:52.771904+00:00",
"revisions": 1,
"comparableWithPrevious": true,
"targetStatus": "missed"
},
{
"id": "7804621f6b59d526c878a7279b9a81d611242021",
"day": "2026-08-12",
"value": null,
"unit": "ratio",
"provenance": "measured",
"coverage": {"cases": 0, "decided": 0, "rate": null, "invalid": 0, "asOf": "2026-08-18T02:14:07.541940+00:00"},
"unavailableReason": "no_data_in_window",
"unavailable": {"reason": "no_data_in_window"},
"definitionFingerprint": "a8bb2e3b1a48b2d8",
"asOf": "2026-08-18T02:14:07.540138+00:00",
"computedAt": "2026-08-13T02:10:31.118442+00:00",
"revisions": 0,
"comparableWithPrevious": true,
"targetStatus": "unknown",
"…": "…"
},
"…",
{
"id": "fa740ee4144e8d007a887f42d1357baac7cd7d37",
"day": "2026-08-18",
"value": 0.75,
"unit": "ratio",
"provenance": "measured",
"coverage": {"cases": 4, "decided": 4, "rate": 1.0, "invalid": 0, "asOf": "2026-08-19T02:12:19.985752+00:00"},
"unavailableReason": null,
"unavailable": null,
"definitionFingerprint": "a8bb2e3b1a48b2d8",
"asOf": "2026-08-19T02:12:19.985752+00:00",
"computedAt": "2026-08-19T02:12:19.985752+00:00",
"revisions": 0,
"comparableWithPrevious": true,
"targetStatus": "met",
"…": "…"
}
],
"breaks": [],
"definitionChangedAt": null,
"trend": {
"direction": "improved",
"current": 0.75,
"previous": 0.3333333333333333,
"delta": 0.4166666666666667,
"pctDelta": 125.00000000000003,
"fromDay": "2026-08-11",
"toDay": "2026-08-18",
"lookbackDays": 7,
"unavailable": null
},
"truncated": false
}| Field | What it tells you |
|---|---|
days | The window actually served, in complete days. Ask for more than the workspace allows (a year, unless yours is set lower) and this is the number you got — with truncated: true beside it, so a clamp never passes for “that’s all the history there is”. |
points | The recorded days, oldest first, one per calendar day. A day that was never recorded simply isn’t there; a day recorded with no number is there with value: null. |
unit · direction · target | The KPI’s own vocabulary, echoed so a chart can label and threshold the series without a second call. |
breaks · definitionChangedAt | Where the meaning of the number changed. Empty and null when it never did. |
trend | The block documented above — always present here, unknown included. |
And per point:
| Field | What it tells you |
|---|---|
day | The complete UTC day this covers. |
value | What the measure came out as that day. null when it produced nothing — never 0. |
unavailableReason · unavailable | Why there is no value. The same vocabulary the live tile uses, so a point and a tile never give two accounts of one absence. |
targetStatus | met · missed · unknown, judged for that day against the KPI’s current target. unknown whenever the day has no value, or the KPI has no target. |
coverage | How many cases that day held, and how many the definition could decide — the same block as on a live read. A 90% containment over four decided cases is not the same fact as 90% over four hundred. |
provenance | measured or inferred, for that day. Promoting a measure never upgrades its evidence, and neither does recording it. |
definitionFingerprint · comparableWithPrevious | The definition behind the number, and whether this point may be compared with the one before it. |
measureVersions | Which versions of your definitions, custom measures and price table were in force. It is what makes a past number explainable a quarter later. |
asOf · computedAt · revisions | When it was last re-checked, when it was first recorded, and how many times the fact actually changed. |
id · projectId · kpiId · measureKey | Identity. The id is derived from the agent, the KPI and the day, which is why re-recording a day rewrites it instead of appending. |
?days must be a positive integer — days=0 is a 422, not an empty chart:
{"detail": "days must be a positive integer."}Record history now
Both routes run the same recording pass Neens runs for you, so a manual catch-up and the automatic one can never disagree. Recording needs the same authority as any other change to a KPI (admin); reading history needs only read access.
curl -sf -X POST "$NEENS_BASE_URL/api/kpis/$KPI_ID/snapshot?days=14" \
-H "Authorization: Bearer $SESSION_TOKEN"{"projectId": "proj_x", "kpis": 1, "days": 14, "computed": 14, "written": 14,
"revised": 0, "skipped": 0, "errors": 0, "truncated": false}| Field | What it tells you |
|---|---|
kpis | How many active KPIs were in scope. Drafts and archived KPIs are not recorded — a draft is not a commitment, and an archived one is over. |
days | How many distinct days the pass covered. |
computed · written | (KPI, day) pairs measured, and rows written. |
revised | Rows whose stored fact actually changed — the late-outcome corrections described above. |
skipped · truncated | What a large catch-up left for the next pass. truncated: true means the backlog was bigger than one pass, and the oldest, stalest days went first; call it again to keep draining. A cap that said nothing would read as “everything is covered”. |
errors | KPIs that could not be recorded. One bad KPI never aborts the rest. |
Omit days and each KPI gets the window it needs: the last few days for one that already has
history (enough to absorb late outcomes), and a one-time backfill of the recent past for one being
recorded for the first time. Pass days explicitly when you need a specific stretch redone —
after correcting a definition, or after a late bulk import of outcomes. It must be between 1 and
the workspace’s ceiling:
{"detail": "days must be between 1 and 365."}Backfill only reaches as far as your data does. Recording a day is a measurement of that day,
not a reconstruction of it: days before your outcomes started arriving are recorded honestly as days
with no number and an unavailableReason, which is the correct answer and a much better one than a
flat line at zero.
How long history is kept
Recorded days are kept for as long as your workspace’s data-retention window keeps anything else, and are aged out with it. On a workspace set to keep data forever (the default), history is kept forever.
Two things worth planning around:
- A shorter retention window shortens your trends. If you keep 30 days, a year-long KPI chart is not available — not because Neens forgot to record it, but because the workspace asked for it to be removed. Export what you need to keep beyond the window.
- Archiving a KPI does not delete its history. The recorded days stay readable, which is most of the point of having promised something: “we committed to 85%, here is what actually happened, here is when we retired the commitment” is the story a KPI exists to be able to tell.
Where your KPIs show up
A KPI is a commitment, so Neens carries it to the places people already look — not just the Business KPIs page. Every surface below reads the same value, target status and trend as the page, through the same request-free resolution path, so a KPI can never say one thing in an email and another on the page — including why a number is missing.
-
Dashboards. The widget gallery grows a Business group — containment rate, containment by agent, resolution time (p50/p90), cost per case, and cost per case over time. See Dashboards. You can slice by agent, status, source, and the metadata on the outcome (its queue, region, plan tier) — so “is containment worse in billing than in shipping?” is a slice, not a new instrumentation agent.
-
The Executive digest dashboard. The shipped Executive persona dashboard now leads with Containment rate and Cost per case — the two numbers an exec reads first. An agent that hasn’t configured those definitions yet sees them render unavailable, never a fabricated
0. -
The digest email. Your daily/weekly digest email carries a Business KPIs section under the fleet-health tiles: your top few active KPIs by priority, each with its value, whether the target is met / missed / unknown, and a trend arrow. A KPI with no number yet shows —, never
0or a red missed. -
The weekly narrative. The weekly “this week in review” narrative now folds your active KPIs into its week-over-week movers, oriented by each KPI’s direction — so a containment slip or a cost-per-case rise surfaces in prose alongside the fleet-health metrics.
-
The Assistant and MCP. Ask the Assistant “how are our business KPIs doing?” or call the
get_business_kpisMCP tool: both return your KPIs with their value, target status and trend, and — for one KPI — the failure clusters eroding it. Anullvalue staysnulland an unknown status staysunknown— the agent is never handed a fabricated number. -
Alerts. Two shapes of alert rule watch a KPI:
- a threshold rule on the measure — “page me when containment drops below 85%”
(
containment_rate,resolution_time_p50/p90andcost_per_caseare first-class metrics); and - a KPI trend rule — “page me when this KPI has regressed by 5% or more over the last 7 days” —
which watches the KPI’s own recent history rather than a fixed line, and
fires only on a genuine regression (an
unknowntrend never pages anyone).
Test a threshold rule against live data before you trust it: containment is measured over decided cases, so a rule on an agent with 8% coverage is watching a very small sample, and the test shows you that now rather than at 3am.
- a threshold rule on the measure — “page me when containment drops below 85%”
(
A widget scoped across agents that define a KPI differently reports unavailable, not an
average. If one agent reads a helpdesk escalation outcome and another reads a classifier label,
one blended containment rate would describe nothing. Neens refuses and names the fix: scope the
widget to one agent, or give those agents the same definition. Two agents that both kept the
platform default still agree — saving the default is not a change of meaning.
Not included (yet)
Worth stating plainly so you don’t go looking:
- Pre-prod eval gates and the
neens evalgate policy gate on a run’s judge metrics, not on catalogue measures. There is nocontainment_rategate rule. - A KPI screen for editing. Promoting, editing and archiving commitments is API-only for now; the values behind them are already on the Business KPIs page, on dashboards, and in the surfaces above.
Late-arriving outcomes
The time window selects cases, by the start of their first trace. Outcomes are joined regardless of when they arrived. That is exactly what makes late outcomes work: a ticket closed three days after the conversation still decides that conversation’s case, and lands in the window the case belongs to.
The consequence: the same window, re-queried later, can show a higher decided count and a
different rate. Nothing changed retroactively — more evidence arrived about the same cases. That
is why every KPI carries coverage.asOf, and why a screenshot of a business KPI is only
interpretable with it.
It is also why recorded history re-checks its most
recent days instead of freezing each one the night it ends: a day’s figure has to be allowed to
catch up with the evidence about that day, and revisions on the point says when it did.
For cost per case, both sides are bounded by the window: a span that started outside the window is not counted toward its case’s cost.
All timestamps on both sides are normalized to UTC. There is no local-timezone arithmetic anywhere in this path.
When Neens shows nothing
A blank is a result. None of these render a zero.
| Situation | What you see |
|---|---|
| No definition configured | The platform defaults, marked not configured, with a Define what counts as escalation call to action. |
| No snapshot for the window yet | — on the tile, with “Value appears once snapshots exist”. The match coverage is in the Data health strip above. |
| The value could not be read | — with “Couldn’t load current values” — a failed read, explicitly distinguished from “no snapshot yet” so you don’t wait on a number that isn’t coming. |
| No decided cases in the window | —, plus “No measured outcomes in this window.” |
| No cases at all in the window | —, plus “No cases in this window.” Never 0, never $0.00. |
| Classifier mode, asking for resolution time | Unavailable, with the reason: a classifier records no resolution instant. |
| Some models unpriced | cost_per_case renders with a partial badge; the value is a floor. |
| Agents in scope disagree on the definition | Unavailable, naming the fix (scope to one agent, or align the definitions). |
| Your only outcome kind is resolution | Containment reports no coverage — a resolution says nothing about whether the agent handled the case. |
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
Containment is exactly 0% | The escalation signal you declared isn’t in your data, so every decided case reads as escalated. | On the Definitions tab, check the picker — a kind or label marked not seen yet decides nothing. |
Containment is exactly 100% | Nothing in your data matches the escalation signal. | Same check. In classifier mode, confirm the escalation labels are the exact strings the classifier emits — they’re matched literally, including case and spacing. |
Containment is — with a real feed | The kinds you send don’t decide containment: only containment/escalation signals do, and only with a readable yes/no. A resolution-only feed decides nothing here. | Send a containment or escalation fact, or point the definition at the kind that carries it. |
| The number looks right but coverage is tiny | Your outcome match rate is low, or your feed only covers some cases. | Data sources → match coverage. |
| Resolution time is unavailable | The KPI is in classifier mode, or its duration kind isn’t a duration. | Switch resolution time to outcome mode and point it at a duration kind. |
| Cost per case is lower than expected | Some models are unpriced, so the ratio is a floor. | Set prices — see Cost & model pricing. |
A tile shows — on a new agent | No snapshot yet for the window. | Nothing to fix — it fills in as snapshots accrue. For a number today, see Cost optimization. |
| A tile says the value couldn’t be loaded | The read failed — a permissions problem or a transient error — as opposed to there being no snapshot. | Reload; if it persists, check that your role still has read access to the agent. |
| The Definitions tab is read-only for you | You’re not an admin. | Ask an admin to declare the definition. |
| A cross-agent widget says unavailable | The agents in scope define the KPI differently. | Scope the widget to one agent, or align the definitions. |
A KPI reads targetStatus: "unknown" | Either the measure produced no value this window, or the KPI has no target yet. Both are unknown on purpose — never missed. | Read unavailable on the response: it names which one. If it’s coverage, the fix is upstream; if it’s the target, PATCH a figure onto it. |
A KPI reads measure_removed | The custom measure it was promoted from has been deleted. | Re-create the measure, or archive the KPI. The commitment is not silently reporting a wrong number in the meantime. |
A KPI reads measure_redefined | A custom measure with the same key exists, but it is not the one this KPI was promoted from — the slug was freed by a delete and then reused for a measure that means something else. | Restore the original definition and the KPI resumes scoring by itself, or archive the KPI and promote the new measure deliberately. A target agreed for one measure is never scored against a different one. |
trend is null with no_snapshots | Nothing has been recorded for this KPI yet — it was promoted a moment ago, or it is still a draft. | Set it active and wait for the next recording pass, or POST /kpis/{id}/snapshot to record it now. |
trend is null with insufficient_history | Fewer than two comparable days a week apart carry a value. | Wait. A KPI promoted on Monday gets its first trend the following Monday; POST /kpis/{id}/snapshot?days=… backfills what your data can support. |
trend is null with definition_changed | The definition behind the number changed inside the window, so the two ends measure different things. | Nothing is broken — breaks on GET /kpis/{id}/history names the day. Re-record the window if the change was a correction rather than a redefinition. |
A day in history shows value: null | That day produced no number — no cases, no decided cases, or the measure could not resolve. | Read unavailableReason on the point. It is recorded deliberately: targetStatus is unknown, never missed. |
Yesterday is the newest day in history | Only completed UTC days are recorded — today is still filling up. | Nothing to fix. Use GET /kpis/{id} for today’s live figure. |
| A recorded day’s value changed | Late outcomes landed and the recent window was re-recorded. revisions went up; computedAt did not move. | Nothing to fix — that is the history catching up with the evidence. |
GET /kpis/{id}/history says truncated: true | You asked for more days than the workspace serves in one call. | Read days on the response for what you actually got. |
POST /kpis returns 409 | Either that measure already has a live KPI, or the agent is at its cap. | The message names which, and which KPI. Edit or archive the existing one. Archived KPIs don’t block a re-commit. |
GET /kpis/summary says truncated: true | There are more active KPIs than the agent’s cap allows — the cap was lowered under existing data, and the tail of the list was dropped. | Archive down to the cap. The flag is there so a shortened scorecard never passes for a complete one. |
A cost KPI reads met but the figure looks low | The window is only partly priced, so the value is a floor — pricing on the response is non-null. | Price the missing models under Settings → Model pricing, then re-read. |
POST /kpis returns 422 on target | A ratio target was typed as a percentage (85 instead of 0.85), or a target went negative on a count/duration/currency measure. | Send the figure in the measure’s own unit — the unit field on GET /kpis/{id} and GET /kpis/options tells you which. |
Related
- Business outcomes — the measured signal behind the Data sources tab, and how to raise your match rate.
- Connect your helpdesk — pull outcomes from Zendesk, Intercom, Salesforce or Jira Service Management on a schedule.
- Derived measures — zero-setup business numbers computed straight from your traces (turns per case, tool success rate, case duration, …), promotable to KPIs.
- Custom measures — define your own KPI when these three aren’t yours.
- Cost & model pricing — where the dollars in
cost_per_casecome from, and what partial means. - Metrics catalogue — the full measure × dimension model these four measures join.
- Issues & failure modes — the failure taxonomy whose labels can also back an inferred KPI.
- Judges — where you turn the ready-made escalation, resolution and sentiment classifiers on, alongside your scorers.
- Fix outcomes — where the fixes that moved this KPI are measured, before vs after each merged fix.
- Dashboards and Alert rules — chart a KPI, or get paged on it.