Insights

Insights are the things Neens notices for you: error-rate and latency anomalies, eval-score regressions, spiking failure modes, novel failure patterns. Detectors run on a schedule, deduplicate what they find, track each signal’s lifecycle, and deliver it where you’ll see it — the in-app Insights page, Slack, an outbound webhook, or a daily digest email.

At a glance

What the detectors watch

Each detector compares a fixed recent 24-hour window against the 24 hours before it, per agent. The thresholds are deliberately simple and documented:

Two more insight types are surfaced on demand rather than by the windowed pass:

• Novel failure — a trace that doesn’t match any known failure cluster (a candidate new failure mode), surfaced by the online cluster assigner.
• Pre-prod regression — a pre-prod evaluation run where the candidate version failed prompts the baseline passed.

The three feeds

The Insights page mirrors the platform’s pillars:

• Observe — a fleet-health line (traces, error %, p99, spend) plus the open anomalies.
• Diagnose — the top problems, ranked and capped at 20: failure clusters, open issues, and score regressions, each linking to the right drill-down. See Failure clustering.
• Fix — ranked remediation recommendations drawn from failure clusters (by confidence × sessions affected, capped at 20), linking to the evidence and to Remediations.

A scope picker narrows all three feeds to all agents, one org, or one agent; the header time picker (default: last 24h) sets the window for the fleet stats and issue rollups. Everything is intersected with your access — asking for an agent you can’t see just drops it.

Refreshing

Detectors run nightly, and the page’s Refresh action (POST /insights/refresh) runs them immediately over your accessible scope — useful right after ingesting new data. Refresh also re-fetches all four sections and asks for a fresh briefing narrative, because new detector output changes the story the briefing is describing.

The fleet briefing

At the top of the Insights page, under the Fleet briefing masthead, is a short newspaper-style lede over the current window: a headline, two or three sentences, and four KPI tiles — Traces, Error rate, p99 latency, Spend.

There are two ways that lede can be written, and the page always shows you which one you’re reading:

• a computed briefing, assembled from the real numbers with no model involved. It is always available, it is never blank, and it is what you see first, every time;
• an AI-written briefing over the same numbers, produced by your agent’s LLM connection in the background and swapped in when it’s ready.

What loads when

Nothing on the page waits for anything else. Each of the four sections is on its own clock and paints the moment its own data lands, behind a placeholder shaped like its real content.

Telling an AI briefing from a computed one

The by-line under the article says so explicitly — different icon and different words:

Two more qualifiers can appear beside it: Updated <when> (when the text was generated) and cached (the text was served from storage rather than composed during this request — including a computed briefing that was stored earlier).

Why a briefing you’d disagree with is never shown

The briefing is stored with a fingerprint of the signals it describes — the fleet KPIs plus the identity of the top anomalies, issues and remediations. On every page load Neens recomputes that fingerprint and compares.

If it doesn’t match, or the stored text is older than the freshness window (10 minutes by default), the AI text is not shown. You get the computed briefing instead, and a fresh narrative is requested in the background.

This is deliberate. A narrative that says “error rate is holding steady” directly above a tile reading 22% is worse than a plain sentence of arithmetic: it teaches you to distrust the page. So a briefing that no longer matches its numbers is downgraded, never displayed.

Driving the briefing over the API

Three routes. GET /insights/summary is the one you read; the other two exist to ask for the AI narrative and to watch it being written.

curl -G https://your-neens/api/insights/summary \
  -H "Authorization: Bearer nk_live_…" \
  --data-urlencode "range=24h"

{
  "headline": "12,480 traces · 4.1% errors · 3 open issues",
  "body": "The fleet ran 12,480 traces at a 4.1% error rate, p99 latency of 3,180 ms and $41.62 in spend. Top anomaly: error rate moved from 0.021 to 0.058 on Support Agent. The leading issue is \"Unsupported order-status claims\" (214 affected).",
  "llm": false,
  "model": null,
  "empty": false,
  "stats": {
    "sessions": 12480,
    "errorRate": 0.041,
    "p99Ms": 3180,
    "spendUsd": 41.62,
    "anomalyCount": 1,
    "issueCount": 3,
    "remediationCount": 2
  },
  "cached": false,
  "generatedAt": "2026-07-30T08:14:02.418Z",
  "briefing": {
    "id": null,
    "status": "missing",
    "refreshing": false,
    "source": "deterministic",
    "signalDigest": "9c41ab77e2d0f3b18a4e5d62",
    "error": null
  }
}

curl -X POST https://your-neens/api/insights/summary/briefing \
  -H "Authorization: Bearer nk_live_…" \
  -H "Content-Type: application/json" \
  -d '{"range": "24h", "from": null, "to": null, "projectIds": null}'

{
  "briefing": {
    "id": "brf_4b1e77c0a9d2f5386e0c14ab",
    "status": "queued",
    "refreshing": true,
    "source": "deterministic",
    "signalDigest": "9c41ab77e2d0f3b18a4e5d62",
    "error": null,
    "headline": null,
    "body": null,
    "model": null,
    "generatedAt": null,
    "stats": null
  }
}

curl https://your-neens/api/insights/summary/briefing/brf_4b1e77c0a9d2f5386e0c14ab \
  -H "Authorization: Bearer nk_live_…"

{
  "briefing": {
    "id": "brf_4b1e77c0a9d2f5386e0c14ab",
    "status": "ready",
    "refreshing": false,
    "source": "llm",
    "signalDigest": "9c41ab77e2d0f3b18a4e5d62",
    "error": null,
    "headline": "Order-status errors double on Support Agent overnight",
    "body": "Error rate on Support Agent jumped from 2.1% to 5.8% across 12,480 traces, driven by 214 sessions making unsupported order-status claims. p99 latency held at 3.2 s and spend was $41.62. Two remediations are already waiting for review.",
    "model": "claude-sonnet-4-5",
    "generatedAt": "2026-07-30T08:14:09.902Z",
    "stats": {
      "sessions": 12480,
      "errorRate": 0.041,
      "p99Ms": 3180,
      "spendUsd": 41.62,
      "anomalyCount": 1,
      "issueCount": 3,
      "remediationCount": 2
    }
  }
}

Briefing statuses

briefing.status is the whole client contract. There are seven values:

Two extra fields matter. refreshing is true when a newer narrative is being written behind the text you were just served — that’s the cue for the small “writing a fuller briefing…” note; keep showing what you have and poll. And source (llm or deterministic) is the only honest way to know who wrote the text you are holding: do not infer it from status, because a computed briefing can be stored and served exactly like an AI one.

When generation fails

A failed briefing always carries a short, fixed reason code in error — never a raw error message. The computed briefing stays on the page throughout.

Note that “this agent has no LLM connection” is not in that list. It is not a failure — see below.

No LLM connection is a supported end state

Neens holds no platform API key. Every LLM feature — the briefing included — runs on the connection you configure under Settings → Connections, on your own account and your own spend.

An agent with no connection gets the computed briefing, permanently. briefing.status reports unavailable, no generation is ever queued, nothing is retried, and nothing is logged as broken. The page is complete: real headline, real body, real numbers, honest by-line. Adding a connection later is all it takes for the AI narrative to start appearing — there is nothing else to switch on.

Lifecycle: dedup, recurrence, resolution

Every insight carries a stable identity (agent + signal, never the run), so re-detection updates one row instead of stacking duplicates:

• First detection ⇒ new. Detected again after being resolved ⇒ recurred. An acknowledged insight stays acknowledged while its signal persists.
• When a detector-owned signal stops firing, the insight is automatically marked resolved on the next pass — feeds only show open insights.
• Feedback survives re-detection because it’s keyed to the same identity.

Feedback and muting

Each insight accepts a verdict — useful, actioned, not_useful, wrong, or dismissed. A negative verdict (not useful / wrong / dismissed) mutes the insight: it disappears from the feeds and stops notifying, until the signal fully resolves and then recurs fresh — at which point it surfaces again as a genuinely new occurrence. Positive feedback is kept for ranking and audit and never suppresses anything.

Persona-aware routing

Detectors stamp every insight with an intended audience (engineer, PM, exec, support) and routing tags (reliability, latency, cost, quality, compliance, safety, pii, issue). Regressions on safety-flavored metrics (toxicity, PII leakage, bias, hallucination, …) automatically pick up compliance/safety tags.

Persona lenses use these to decide which insights lead for a user: an Executive lens leads with reliability/quality/cost signals, Finance with cost, Compliance with compliance/safety/PII signals and pre-prod regressions, Product manager with quality regressions. Catch-all lenses (developer, admin, full workspace) see everything. Routing shapes the digest email below — the in-app feeds always show the full picture, and a lens never hides data you have access to.

Notification sinks

When an insight newly surfaces (first appearance, or recurrence after resolve/mute), it’s dispatched to the configured sinks:

Delivery is best-effort: a down Slack or webhook endpoint is logged and never breaks detection.

The digest email

The digest is an opt-in, per-user email — “your fleet, every morning”:

• Opt in under Settings › Account (or PUT /auth/digest), choosing daily or weekly. It’s off by default, and capped at one email per day per user.
• Contents: the fleet-health tiles — Traces, Error rate, Eval pass rate, Spend — computed through the same metrics catalogue the dashboards use, plus up to five open “needs attention” insights routed to your persona lens, each deep-linking into the app. When your agent has committed to Business KPIs, the digest adds a Business KPIs section — your top few by priority, each with its value, target status (met / missed / unknown) and trend. A KPI with no number yet shows —, never a zero.
• Scope-safe: every number and insight is limited to the agents you could see in the app.
• Send time: the daily job fires at 13:00 UTC. Weekly recipients get theirs on the same tick once a week.
• Never noisy: a digest with nothing to say (no activity, no insights) is skipped, not sent empty.
• Try it: Send me a test digest in Settings (or POST /auth/digest/test) composes and sends yours immediately, ignoring the daily cap.

Troubleshooting

Related

• Dashboards — build the charts these signals point you at.
• Metrics catalogue — the measures behind the fleet stats and digest tiles.
• Failure clustering and Remediations — where insight links land.