Fix bundles

A fix bundle is a remediation exported as a single, self-contained document you hand to a coding agent. It packages everything Neens already worked out about one recurring failure — the root cause, the concrete fix, a few anonymized failing examples, and a ready-to-run proof-eval command — into one block of Markdown you paste into your own Claude Code / Cursor / Codex session to implement the change and gate the pull request.

Neens does the diagnosis; your coding agent, in your repo, with your credentials, does the edit. The bundle carries no keys and needs no repo access — see Zero new trust surface.

At a glance

Review and copy a fix bundle

Nothing is copied behind your back: you open the bundle, read the exact Markdown that will be copied, and then put it on your clipboard yourself.

What’s in the bundle

The bundle is a Markdown document with five sections:

1. Problem & root cause — the failure cluster’s label and description, the root-cause hypothesis and where to look, and how many failing sessions it was observed in.
2. Proposed fix — the remediation’s typed artifact: the target (what to change, e.g. a system prompt or a tool schema), the change instruction and rationale, and the concrete before/after (or a unified diff).
3. Failing examples (anonymized) — two to three real production traces that exhibit the failure, with input/output/error. Every free-text field is redacted before it leaves your tenant (see the callout below).
4. How to verify (proof evals) — a ready-to-run neens eval run command plus a neens-gate.json gate-as-code policy, so CI can replay the failure against your fixed agent and block the PR on a regression.
5. Acceptance criteria — a short “done” checklist grounded in the actual gate, judge, dataset, and failing examples.

The bundle matches the fix’s locus

A bundle is only paste-into-your-agent-shaped when the fix actually is an agent change. For a remediation whose failure locus isn’t your agent, the bundle is rendered honestly so you don’t hand a coding agent a prompt edit for a problem a prompt can’t fix:

• Infra advisory / escalate to owner — the bundle leads with “This is a downstream/upstream service failure, not an agent defect,” names the owning component, states whether the agent already handles the error (retry + backoff), and gives the recommended resilience change. Its “how to verify” is a resilience / fault-injection check, not a judge eval.
• Working as intended — the bundle says “No action recommended — a control fired correctly,” with the evidence (the guardrail and the rule it enforced). There’s no diff to apply.
• Needs grounding — the bundle says “Not yet actionable,” gives the reason, and suggests the next step (re-cluster or gather more evidence) rather than a code change.

For an actionable prompt or tool fix, the bundle keeps the full paste-ready format described above.

Gate the pull request in CI

Section 4 of the bundle contains a pre-generated command for the neens eval CI runner. When the remediation is linked to a proof-eval gate — a failure mode bound to a dataset and a judge — the command already references the real dataset, so it’s ready to run:

neens eval run --create --dataset <dataset-id> \
  --version-label "$GIT_SHA" \
  --min-pass-rate 0.90 \
  --gate-policy neens-gate.json \
  -- <your agent command>

Replace <your agent command> with how CI invokes your agent, and save the neens-gate.json policy from the bundle alongside it. The process exit code is the gate verdict — 0 means the fix held — so your pipeline gates on it directly. The default --min-pass-rate is 0.90, matching how Neens scores production. See Pre-prod evaluations for the full CI runner.

Zero new trust surface

A fix bundle is the Neens diagnosis rendered into context for your tools. Building it:

• never calls an LLM — it’s a deterministic assembly of data Neens already computed from your traces;
• never touches the network and never reads your repository;
• carries no credentials — the coding agent runs in your environment, on your repo, under your review.

You paste it into the coding agent you already trust, so applying a fix adds no new integration, access grant, or outbound connection.

How it works

The bundle is assembled read-only from data the remediation already carries:

• The root cause comes from the linked failure cluster’s stored analysis.
• The fix is the remediation’s typed artifact (the same before/after you review in the panel).
• The failing examples come from the evidence snapshot captured when the remediation was generated (Neens falls back to a fresh read of the cluster’s member sessions if that snapshot is empty), each field redacted.
• The proof gate resolves the flywheel eval gate bound to the remediation’s failure mode — its dataset and judge — to fill in the neens eval run command and acceptance criteria.
• The repo link, when present, comes from your agent’s optional agent context (Settings the remediation reads for grounding).

Because it’s pure assembly, the same remediation always produces the same bundle.

Reference

Troubleshooting

• “Couldn’t build the fix bundle. Try again in a moment.” — the bundle request failed. It’s usually transient: close the dialog and click View fix bundle again. If it keeps failing, the remediation may have no usable content to assemble (e.g. no fix artifact and no linked cluster) — generate or complete the remediation first (Remediations).
• The dialog says “Nothing to show.” — the bundle was built but came back empty. Complete the remediation (a fix artifact or a linked failure cluster) and open it again.
• “Couldn’t copy automatically — select the text and copy it manually.” — your browser denied clipboard access (common in embedded or non-HTTPS contexts). Select the Markdown in the dialog and copy it with your keyboard; the text is the complete bundle.
• There is no View fix bundle button — fix bundles are turned off for this deployment; ask whoever runs your Neens instance.
• The command shows <dataset-id> — the remediation isn’t bound to a proof dataset yet. Run Generate eval on its failure mode (Eval gates) and open the bundle again.