MCP server

Neens exposes a Model Context Protocol (MCP) server so any MCP-compatible agent — Claude Desktop, an IDE assistant, or your own automation — can both read and write the failure→fix loop with tools instead of hand-written HTTP calls. This is the foundation of the “agents that fix agents” workflow: an agent can inspect what’s failing, curate a dataset, record a ground-truth label, stand up a judge, and move a remediation forward — all through the same permission and tenancy model your team uses.

At a glance

Connect

The server speaks MCP over Streamable HTTP at POST <base-url>/mcp on the same origin as the app. Every client needs the same two things: the endpoint URL and your agent key sent as a bearer token (Authorization: Bearer nk_live_…). Pick your client below.

Your agent API key is shown in Settings → API keys (and is printed by the seed scripts for local demos). It resolves to exactly one company + agent, so the agent can only ever see and change that agent’s data.

claude mcp add --transport http neens \
  https://YOUR_NEENS_HOST/mcp \
  --header "Authorization: Bearer nk_live_your_project_key"

{
  "mcpServers": {
    "neens": {
      "url": "https://YOUR_NEENS_HOST/mcp",
      "headers": { "Authorization": "Bearer nk_live_your_project_key" }
    }
  }
}

Endpoint:      https://YOUR_NEENS_HOST/mcp   (POST, JSON-RPC 2.0)
Authorization: Bearer nk_live_your_project_key

{
  "mcpServers": {
    "neens": {
      "url": "https://YOUR_NEENS_HOST/mcp",
      "headers": { "Authorization": "Bearer nk_live_your_project_key" }
    }
  }
}

Verify the connection. Once connected, ask the client to list the Neens tools (an MCP tools/list call). You should see 52 tools — 34 read and 18 write — described in What an agent can do. Seeing none means the connection isn’t working — re-check the URL and that the Authorization header is being sent.

// tools/call get_trace {"trace_id": "<a session in another project>"}
{
  "error": "This credential is not permitted to perform this action (project keys can read + write within their project, but cannot delete; some actions require a member/admin role).",
  "status": 403
}

What an agent can do

The 52 tools are grouped below by the job they do, in roughly the order a loop uses them. Every group is labelled Read or Write per tool: a read tool carries MCP’s readOnlyHint annotation so a host can auto-approve it, a write tool doesn’t. No tool deletes anything.

Your connection

An agent API key and an SSO/OAuth MCP token each pin one agent, so get_current_project is the direct answer to “which agent am I connected to?”. Call it at the start of a session — for example right after signing in through SSO — to confirm you landed in the agent you expected before reading or writing anything:

// tools/call get_current_project {}
{
  "projectName": "Help agent",
  "projectId": "project_7c2a…",
  "orgName": "Support",
  "companyName": "Acme",
  "credentialKind": "user",   // "project" for a project API key; "user" for an SSO/OAuth token
  "role": "member",
  "scopePinned": true          // the agent is fixed for this connection
}

The server also names the scoped agent in its initialize reply’s instructions, so a capable client knows the agent from the handshake without an extra call. The tool never reveals another agent: the answer comes from your own credential, and a null result simply means no single agent is pinned (a user session that selects its active agent per request).

Traces and sessions

Failure modes and clusters

Business KPIs

Scores, judges and eval runs

One read is enough to deploy and run

list_judges is the only call you need before either write tool. Each judge it returns carries the version ids and the deployment ids, so nothing has to be looked up in the UI and nothing has to be guessed:

{
  "id": "judge_a41f",
  "name": "Grounding check",
  "currentVersionId": "jver_9c02",
  "latestVersionId": "jver_9c02",
  "latestVersionNum": 2,
  "versions": [
    {"id": "jver_9c02", "versionNum": 2},
    {"id": "jver_31ab", "versionNum": 1}
  ],
  "deployments": [
    {"id": "dep_5e7d", "versionId": "jver_9c02", "trigger": "manual", "status": "enabled"}
  ]
}

versions is newest-first and holds up to the 5 most recent; latestVersionId is always the newest one. A judge with no deployments entry whose status is enabled has nothing running — that is the one to call deploy_judge on. (It may still have disabled entries: those are deployments that were turned off, not judges that were never deployed.)

Deploying without a version id

judge_id is the only required argument. Omit version_id and the judge’s latest version is resolved for you:

{"judge_id": "judge_a41f", "trigger_policy": "manual"}

A version number works too, so {"judge_id": "judge_a41f", "version_id": "1"} deploys version 1 — you don’t have to translate it into jver_31ab yourself. Version ids are matched before numbers, so an id made only of digits still wins.

The reply names what was actually deployed, which matters when you let the server pick:

{
  "id": "dep_5e7d",
  "judgeId": "judge_a41f",
  "versionId": "jver_9c02",
  "resolvedVersionId": "jver_9c02",
  "trigger": "manual",
  "status": "enabled",
  "successThreshold": null,
  "alreadyDeployed": false
}

If a version genuinely can’t be resolved, the error lists the judge’s real version ids (Its versions are: version 2 = jver_9c02; version 1 = jver_31ab) so the next call succeeds. Never retry deploy_judge with an invented id.

Running an eval from a judge id

run_eval takes deployment_id, but if you only have the judge, pass that instead and its single enabled deployment is used:

{"judge_id": "judge_a41f"}

The reply carries resolvedDeploymentId alongside the run id, status and total, so you always know which deployment actually ran. Two cases are refused rather than guessed at:

By default a run covers what the deployment’s sampling filter selects. Pass sample_size to cap this run at the N most-recent eligible traces without editing the deployment — e.g. sample_size=1000 to score 1000 traces. The response total is the number actually queued, which is smaller when fewer traces are eligible. success_threshold independently overrides what counts as a pass for that one run.

Datasets, versions and exports

Annotations

Pre-prod evaluations

Use these to run a pre-prod evaluation yourself. When you are proving one specific remediation’s fix, run_verification in the fix loop does the same job in one call.

Comparing results

The remediation backlog

The fix loop

The six tools that let a coding agent pull a fix, prove it, and report the result back — see Fix loop over MCP for the round trip.

Orchestrated fix runs

Prompt optimization

Model sweeps

Exports return a manifest, not the file

export_dataset and export_dataset_version deliberately never put the export body in the tool result. A real golden export is far past the 20,000-character result cap, so inlining one would go through the shrinking described below and hand you a file that is missing rows but still parses as valid JSON. An export you cannot tell is incomplete is worse than no export tool, so you get everything needed to act on it — and a URL for the bytes:

{
  "datasetId": "ds-7f2a91",
  "datasetName": "checkout-timeout-v1",
  "format": "json",
  "rowCount": 4821,
  "sampleFields": ["id", "kind", "traceId", "sessionId", "input", "expectedOutput", "output"],
  "csvColumns": ["kind", "trace_id", "session_id", "input", "expected_output", "output"],
  "itemsChecksum": "sha256:9d31c0…",
  "bytes": 3841902,
  "sampleRowCount": 3,
  "sample": [ /* the first 3 rows, each field capped at 300 characters */ ],
  "complete": false,
  "retrieval": {
    "method": "GET",
    "path": "/api/datasets/ds-7f2a91/export",
    "csvPath": "/api/datasets/ds-7f2a91/export?format=csv",
    "note": "…"
  }
}

To fetch the full export, call the retrieval.path over ordinary HTTP with the same bearer token you connected the MCP server with — no extra credential, and the same agent scope:

curl -H "Authorization: Bearer nk_live_your_project_key" \
  "https://YOUR_NEENS_HOST/api/datasets/ds-7f2a91/export" -o dataset.json
 
# …or as CSV
curl -H "Authorization: Bearer nk_live_your_project_key" \
  "https://YOUR_NEENS_HOST/api/datasets/ds-7f2a91/export?format=csv" -o dataset.csv

If you only need a few rows and not a file, get_dataset_items (or get_dataset_version_items) pages them properly and reports total alongside returned.

Walkthroughs

Three end-to-end flows you can run today. Each shows the prompt you’d type at your coding agent and the wire traffic underneath, so the same recipe works whether you’re driving a chat client or writing the JSON-RPC by hand.

Triage a failure mode and curate a golden dataset

The everyday loop: find what’s failing, look at real examples, record verdicts, and freeze the result into a dataset your judges can evaluate against forever.

Prompt. “Using the neens tools, find the biggest new failure mode in this agent, pull 20 example traces from its failure set, mark the ones that are genuinely broken as fail, and collect them into a golden dataset called checkout-timeout-v1.”

{"cluster_id": "cl_7f2a91"}

{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "list_traces",
    "arguments": {
      "failure_set": true,
      "issue_mode": "Checkout tool timeout",
      "status": "error",
      "limit": 20
    }
  }
}

{
  "jsonrpc": "2.0",
  "id": 3,
  "result": {
    "content": [{
      "type": "text",
      "text": "{\"total\":83,\"traces\":[{\"id\":\"sess_4d19\",\"agentName\":\"checkout-agent\",\"status\":\"error\",\"startedAt\":\"2026-07-28T09:14:22Z\",\"turnCount\":6,\"failureMode\":\"Checkout tool timeout\"}]}"
    }],
    "isError": false
  }
}

{
  "target_type": "session",
  "target_id": "sess_4d19",
  "verdict": "fail",
  "critique": "Tool call to /checkout exceeded the 30s budget; agent retried instead of failing over.",
  "is_gold": true,
  "cluster_id": "cl_7f2a91"
}

{
  "name": "checkout-timeout-v1",
  "description": "Traces from the checkout tool-timeout failure mode, July 2026.",
  "session_ids": ["sess_4d19", "sess_51c0", "sess_6b77"],
  "golden": true
}

{"run_id": "run_9c21", "limit": 20}

From a failure cluster to a pre-prod comparison

The full evaluation loop, using nothing but MCP tools: pick a failure cluster, freeze its traces into a golden set, evaluate a candidate against it, and compare the result to a baseline.

Prompt. “Using the neens tools, take the biggest failure cluster in this agent, freeze its traces into a golden dataset, run a pre-prod evaluation of branch fix/checkout-timeout against it, and tell me whether anything regressed versus the current baseline.”

{"cluster_id": "cl_7f2a91", "limit": 5}

{"cluster_id": "cl_7f2a91", "limit": 50}

// create_dataset
{"name": "checkout-timeout-v1", "session_ids": ["sess_4d19", "sess_51c0", "sess_6b77"]}
 
// create_golden_version
{"dataset": "checkout-timeout-v1", "name": "v1", "notes": "Traces from cluster cl_7f2a91"}

{
  "name": "checkout-timeout candidate",
  "dataset_id": "checkout-timeout-v1",
  "version_label": "fix/checkout-timeout@a1b2c3d",
  "runner_mode": "push",
  "agent_connection_id": "conn_preview",
  "min_pass_rate": 0.9,
  "max_regressions": 0
}

{
  "runId": "ppr_9d31",
  "candidate": {"scored": 30, "passed": 27, "passRate": 0.9},
  "regressionCount": 2,
  "newPassCount": 5,
  "matrixTotal": 30,
  "regressions": [ /* capped evidence rows */ ],
  "deltas": {"latencyMs": {"before": 1400, "after": 1180, "delta": -220}}
}

{"run_ids": ["ppr_9d31", "ppr_9d44"], "baseline_run_id": "ppr_9d31"}

{"run_id": "ppr_9d44", "baseline_run_id": "ppr_9d31", "item_id": "dvi_0875bc"}

Close the loop on a fix

The differentiator: your coding agent writes the fix in your repo, and Neens proves it with a real evaluation against your preview deploy.

Prompt. “Pick the highest-priority open remediation from neens, get its fix bundle, implement it on a branch, deploy it to preview, then ask neens to verify the fix at that branch and report the PR back.”

{
  "jsonrpc": "2.0",
  "id": 9,
  "method": "tools/call",
  "params": {
    "name": "run_verification",
    "arguments": {
      "remediation_id": "rem_2c81",
      "version_label": "fix/checkout-timeout@a1b2c3d",
      "endpoint_url": "https://pr-482.preview.example.com",
      "request_shape": "openai_chat",
      "min_pass_rate": 0.9,
      "max_regressions": 0,
      "wait_seconds": 120
    }
  }
}

{"runId":"pre_9d31","status":"completed","passed":true,"reasons":[],"regressions":0,"passRate":0.94,"aggregate":{"scored":50,"passed":47}}

{
  "remediation_id": "rem_2c81",
  "status": "verified",
  "work_state": "done",
  "pr_url": "https://github.com/acme/checkout-agent/pull/482",
  "commit_sha": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0"
}

Fix loop over MCP

The six fix-loop tools let a coding agent do more than investigate: it can pull a fix off the Neens backlog, implement it in your repository, ask Neens to prove the fix works, and report the result back — all without Neens ever touching your code. Neens does the diagnosis and the verification; your coding agent, in your repo, with your credentials, does the edit and opens the pull request. A human still reviews and merges.

This is what sets the Neens MCP apart: other observability MCP servers stop at read-only triage. The Neens run_verification turns “here’s a suggested fix” into “here’s a fix that provably passes the same evals the failure originally broke.”

The round trip

Close the loop on a fix walks the same seven steps with real arguments and the wire traffic.

How it works

Each tools/call builds a short-lived in-process client that re-enters an existing Neens REST endpoint (for example, list_traces calls GET /sessions, add_annotation calls POST /review), forwarding your Authorization header. The Neens tenant resolution and each route’s authorization gate run per call, so there is no separate access path and no way for an agent to exceed the connecting key’s permissions.

The server is stateless and advertises only the tools capability. Reads are available to any agent-scoped credential; writes are admitted for an agent key on the specific non-destructive write routes, staying leak-safe because the key is pinned to a single agent.

Results are compacted, never truncated mid-JSON

A tool result feeds a model’s context window, not a UI, so every result is compacted on the way out. This is visible to you, and it is worth knowing before you build on a response shape.

The result is always valid JSON. If the compacted payload still exceeds 20,000 characters, Neens repeatedly halves every list and re-serializes — up to 20 passes — rather than cutting the string off somewhere arbitrary. Any result that went through that shrinking carries a top-level marker so a client can tell:

{"total": 4821, "traces": [ /* … */ ], "truncated": "…(truncated)"}

So a "truncated" key means there was more — narrow the query (a tighter limit, a time bound, a single id) rather than assuming you saw everything. In the pathological case where there are no lists left to shrink, a successful payload becomes {"truncated": "…(truncated)", "data": "<sliced JSON as a string>"} — still parseable, deliberately obvious. Any top-level error, ref or status it carries stays a real key beside those two rather than going into data. A failed call is handled differently, and never loses its reason: see A truncated failure still tells you why.

”The call failed” vs. “the run I asked about failed”

These are different facts and Neens reports them in different places. Read them in this order:

1. isError on the JSON-RPC result answers did the tool call work?
2. Fields inside the payload — status, error — answer what did I learn about the thing I asked for?

A failed call is never a protocol error and never crashes the session: the reply carries "isError": true and the text block holds {"error": "<message>"}, so the agent can read it and recover. A payload status is present only when the failure came back from the underlying REST route — a non-2xx adds {"status": <code>}. The failures raised before any route is reached (an unknown tool name, a missing required argument, a tool that refuses its own arguments, an executor that threw) carry error alone, so key off isError and treat status as optional. A 403 always returns the same fixed sentence rather than echoing the route’s own message — the credential’s permissions are not something to probe by reading error text.

A payload field named error is data, not a verdict. get_fix_run, get_prompt_optimization, get_model_sweep and their start_… siblings all project the run’s own error string at the top level. It is null on a healthy run and a real message on a run that broke — and in both cases the tool call itself succeeded, so isError is false.

A healthy run — the call worked, and so did the run:

{
  "jsonrpc": "2.0", "id": 7,
  "result": {
    "isError": false,
    "content": [{ "type": "text", "text": "{\"id\":\"fxr_9c21\",\"status\":\"pr_opened\",\"attempt\":1,\"passK\":2,\"prUrl\":\"https://github.com/acme/agent/pull/412\",\"failureReport\":null,\"error\":null}" }]
  }
}

A run that failed — the call still worked, and error is how you find out why:

{
  "jsonrpc": "2.0", "id": 8,
  "result": {
    "isError": false,
    "content": [{ "type": "text", "text": "{\"id\":\"fxr_9c22\",\"status\":\"failed\",\"attempt\":3,\"passK\":2,\"prUrl\":null,\"failureReport\":\"pass^2 not met: 1/2 green\",\"error\":\"patch did not apply to origin/main\"}" }]
  }
}

A call that failed — no payload to read, just the reason:

{
  "jsonrpc": "2.0", "id": 9,
  "result": {
    "isError": true,
    "content": [{ "type": "text", "text": "{\"error\":\"fix run not found\",\"status\":404}" }]
  }
}

How to handle a tools/call reply, in order:

1. If isError is true, the call did not happen — parse error (and status, if present) and decide whether to retry. Retrying is only useful for a 5xx; a 403, a 404 or a missing argument will fail identically forever.
2. Otherwise you have a real payload. If the tool reports on a run (get_fix_run, get_prompt_optimization, get_model_sweep), branch on the payload’s status field, then read error for the reason when that status is a failure. get_verification_run is the exception: it reports a gate rather than a run error, so its detail is passed plus reasons[] and it projects no error field at all.
3. Never treat the presence of an error key as a failure — on a healthy run it is present and null.

get_trace is the one read that spells the field differently: a trace’s own error message rides as errorMessage (with per-tool-call error strings inside toolCalls[]). That name predates the rule above and is kept for compatibility.

An unexpected failure is redacted to a reference id

A tool can fail in a way nobody wrote a message for — the database is unreachable, an upstream connection is refused, a query names a column that isn’t there. Those failures still come back as an ordinary failed call ("isError": true, a text block holding {"error": …}), but the message is deliberately the same fixed sentence every time, plus a ref:

{
  "jsonrpc": "2.0", "id": 11,
  "result": {
    "isError": true,
    "content": [{ "type": "text", "text": "{\"error\":\"the tool failed unexpectedly (ref: 4b1c8ad02e7f)\",\"ref\":\"4b1c8ad02e7f\"}" }]
  }
}

The underlying exception is not relayed. Its message routinely carries the server’s database file path, an internal hostname and port, or the name of a tenant’s ClickHouse database — and a tool result travels straight into an agent’s context window and whatever that client stores, which is a one-way trip. So the detail stays server-side and the ref is the handle to it.

For an agent, ref is what to quote. There is nothing to parse and nothing to probe: treat the call as failed, do not retry it blindly (an unexpected failure repeats until an operator fixes it), and if you report the problem to a human, include the ref verbatim.

The same rule applies one layer up. If the JSON-RPC dispatch itself fails, the protocol error is {"code": -32603, "message": "Internal error (ref: <id>)"} — same handle, same reason.

# container / compose
docker compose logs web | grep 4b1c8ad02e7f
# kubernetes
kubectl logs deploy/neens-web | grep 4b1c8ad02e7f

Failures Neens does have words for are unchanged and still say what to do: an unknown tool name, a missing required argument, a tool refusing its own arguments, a 403, a 404 and every other non-2xx all come back with their own message (and a status, when a REST route produced it). Only the unanticipated ones are redacted.

A truncated failure still tells you why

Compaction and failure reporting are two rules that used to be able to contradict each other. They no longer can, and the guarantee is worth writing your client against:

Whenever isError is true, the text block parses to a JSON object with a top-level error key holding a non-empty string. Truncation can shorten that string — never move it, never drop it.

The reason is the last thing a result gives up. When a failure is too large for the cap, Neens sacrifices in this order: the bulk payload is halved and then demoted into a data string, the truncated marker goes, status goes, and what remains is the reason — shortened with the usual trailing … if it has to be, but never emptied. A redacted failure’s ref (above) is never sliced, because half a correlation id identifies nothing: it survives whole past status, and is given up whole only after the reason has already been shortened. So a large failure looks like this:

{
  "jsonrpc": "2.0", "id": 11,
  "result": {
    "isError": true,
    "content": [{ "type": "text", "text": "{\"error\":\"upstream refused the patch: pass^2 not met on attempt 3 of 3\",\"status\":502,\"truncated\":\"…(truncated)\",\"data\":\"{\\\"raw\\\":{\\\"attempts\\\":[…\"}" }]
  }
}

error and status are real keys you can read directly. truncated says context was dropped, and data is whatever was demoted — a sliced JSON dump carried as a plain string, so treat it as opaque diagnostic text and never as something to parse.

For a client author:

1. Branch on isError first. If it is true, JSON.parse the text block and read error — it is always there and always non-empty, so there is no “unknown failure” branch to write.
2. Read status if present to decide whether to retry (5xx yes; 403/404/a bad argument will fail identically forever). Its absence means the failure happened before any route was reached, not that the status was hidden.
3. Treat truncated as informational: context was dropped and data, if present, is partial. Do not parse data, and do not read the failure out of it. Note that truncated present means the reason is whole — the marker is given up before the message is, so the two never appear abridged together.
4. A reason ending in … is a shortened message, not a different one — it is always a prefix of the full text. It also means every droppable key had already been dropped, so do not read the absence of status as a signal in that case.