What changed

When quality drops, the first question is always “what did we ship just before this?” Neens answers it with a deploy-event ledger: record every prompt, model, tool, or config change your agent goes through, and Neens correlates score regressions and failure spikes to the events that landed in the window just before — closest change first.

It’s a small habit with a big payoff: a failing eval gate or a regression insight stops being a mystery and becomes “pass rate dropped right after prompt v7.”

At a glance


Where	The What changed panel on a remediation’s detail, plus the correlation API for any timestamp
Key API	`POST /deploy-events`, `GET /deploy-events`, `GET /deploy-events/what-changed`
Needs	Nothing but the events themselves — no LLM involved; correlation is a pure time-window computation
Scope	Agent-scoped; events can additionally be tagged with an agent name

Record deploy events

Create an event whenever something about your agent changes — ideally automatically from CI, so the ledger is complete without anyone remembering to write it.

The event shape

POST /deploy-events takes:

Field	Required	Meaning
`kind`	yes	One of `prompt`, `model`, `tool`, `param`, `config`, `judge` — rejected with `422` otherwise
`title`	yes	Human-readable summary, e.g. `"Prompt v7: stricter grounding rules"`
`agentName`	no	Which agent this change applies to (used to filter correlations)
`oldValue` / `newValue`	no	The before/after value — a model id, a prompt version, a parameter
`deployedAt`	no	ISO-8601 timestamp of the deploy; defaults to the time of the API call
`deployedBy`	no	Who or what shipped it, e.g. `"ci"` or a teammate’s name

curl -X POST "https://<your-neens-host>/deploy-events" \
  -H "Authorization: Bearer $NEENS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "model",
    "title": "Switched support agent to new model version",
    "agentName": "support-agent",
    "oldValue": "model-v1",
    "newValue": "model-v2",
    "deployedBy": "ci"
  }'

Record every kind of change, not just code deploys — a prompt tweak, a model swap, a new tool, a temperature change, or an updated judge can each move your metrics. The kind field is what makes the correlation readable later.

Browse the ledger with GET /deploy-events (newest first; filter by agentName and since).

The correlation

GET /deploy-events/what-changed answers “what changed before this moment?” for any timestamp:

curl "https://<your-neens-host>/deploy-events/what-changed?at=2026-07-15T09:00:00Z&agentName=support-agent" \
  -H "Authorization: Bearer $NEENS_API_KEY"

Parameter	Required	Meaning
`at`	yes	The ISO-8601 reference instant — when the regression / anomaly surfaced
`agentName`	no	Restrict to one agent’s events
`windowHours`	no	Look-back window; default `168` (7 days)

The response lists the events deployed within the window before at, closest first, each annotated with hoursBefore — how many hours before the reference instant it shipped. Events after at are excluded (they can’t have caused it), and events with missing timestamps are skipped rather than erroring.

{
  "at": "2026-07-15T09:00:00Z",
  "windowHours": 168,
  "events": [
    {"kind": "prompt", "title": "Prompt v7: stricter grounding rules", "hoursBefore": 6.5, "...": "..."},
    {"kind": "model",  "title": "Switched support agent to new model version", "hoursBefore": 30.25, "...": "..."}
  ]
}

The correlation is deliberately honest about what it is: a time-window correlation, not proof of causation. Its job is to shrink “anything could have caused this” down to one or two candidate changes you can actually investigate.

Where it surfaces in the product

Remediations — every fix’s detail panel includes a What changed section listing the deploy events recorded for the failure’s agent in the 7 days before the fix was proposed, each with its “N h before” distance. If a failure appeared right after a prompt change, you’ll see it next to the proposed fix — and if a fixed failure regresses, the same panel points at the change that likely broke it again. See Remediations.
Eval gates — when a gate’s pass rate drops below its baseline, query the correlation at the run’s timestamp to see what shipped just before. See Eval gates.
Release flow — pre-prod evaluations compare candidate versions explicitly; the deploy ledger complements them by covering everything that ships outside that gated path. See Pre-prod evaluations.

Practical workflow

Tag deploys from CI

Add the POST /deploy-events call to every pipeline that changes your agent — code deploys, prompt updates, model swaps, config pushes. Use a consistent agentName (matching the agent name your traces report) so correlations stay tight.

Watch your quality signals

Let judges score production continuously, keep your eval gates active, and let insight detectors watch for score regressions and failure-volume anomalies.

When something drops, ask what changed

Open the affected remediation’s What changed panel, or hit GET /deploy-events/what-changed?at=<when it started>. The closest-first ordering usually puts the culprit at the top of the list.

Close the loop

Fix forward (or roll back), record that change as a deploy event too, and confirm recovery with a gate run. The failure, the fix, and both deploys now sit on one timeline.

Eval gates — the regression guards that make a drop visible
Remediations — fixes with deploy correlation built into their detail
Insights — automatic anomaly and regression detection
Issues and failure modes — the failure taxonomy behind it all
Judges — the continuous scoring that produces the signals worth correlating

Eval gates Pre-prod evaluations