Autonomy levels
The eval-verified PR proves a fix works before it ships. Autonomy levels decide how much of the path to that PR runs without you clicking anything: at the highest level, a failure can go from ingested trace to an open, proof-carrying pull request with no human action in between.
A human still merges every pull request, at every level. Autonomy removes human gates — who accepts a remediation, who starts the verified run — and never the human merge. Neens does not merge, approve, or deploy anything, and nothing on this page changes that.
At a glance
| Where | Settings → Autonomy (admin-only) |
| Key API | GET /autonomy/status · PATCH /autonomy/settings · POST /autonomy/suspend · POST /autonomy/resume · GET /autonomy/decisions · PUT /spend-budgets/fix_engine |
| Scope | Per agent. Two agents in the same company can sit at different levels |
| Default level | propose_only — today’s behaviour: a human accepts, a human starts the run |
| What it needs | The fix engine working end to end: remediations with proof-eval gates, a VCS installation, and at least one human-aligned verifying judge |
| Merges? | Never. At every level a human reviews and merges the PR |
| Cadence | A sweep runs every 15 minutes |
The four levels
| Level | Who accepts the remediation | Who starts the eval-verified run | Who merges |
|---|---|---|---|
off | (nothing autonomous happens) | — | — |
propose_only (default) | A human | A human | A human |
auto_verify | A human | Neens | A human |
auto_pr | The auto-accept policy | Neens | A human |
In the Autonomy level card these appear as Off, Propose only (default), Auto-verify and Auto-PR.
- Off — no autonomous activity at all. Nothing is accepted and no fix run is started for you.
- Propose only (default) — exactly today’s behaviour. The clustering run pre-populates remediations; a human accepts one, and a human starts the eval-verified fix run.
- Auto-verify — a human still accepts the remediation. That is the whole difference from
auto_pr, and it is easy to oversell: at this level Neens only removes the second click. Once a person has moved a remediation toaccepted, the next sweep starts the verified fix run for it. A PR opens only if pass^k, the accumulated regression set and the κ-anchored verifying judges all hold — unchanged fix-engine behaviour. - Auto-PR — the auto-accept policy accepts an eligible remediation without a human, then the same verified run happens. No human action between ingest and PR.
Raising the level never weakens the proof. auto_verify and auto_pr start the same fix run a
human would have started, through the same validation: the remediation must be in scope, it must
have a resolvable proof-gate dataset, and the same pass^k / regression-set / anti-gaming rules
decide whether a PR opens at all. There is no faster path for a machine-started run.
Turn it on and pick a level
Get the prerequisites in place
At auto_verify and auto_pr an agent needs everything a manual
eval-verified PR needs, plus one thing it can live without manually: a
human-aligned verifying judge. If no judge in the agent has a measured Cohen’s κ at or above
the effective floor (see The κ floor), autonomy suspends itself rather than running unanchored — label
gold items under Annotations & review so alignment is measured.
Choose a level
Open Settings → Autonomy, pick a radio card in the Autonomy level card, and press Save. Every card restates what stays human, including the merge.
Start at Auto-verify. It is the level where you keep the judgement call (“is this the right fix?”) and hand over only the mechanical one (“go prove it”). Move to Auto-PR once the decision log shows you agree with what the policy would have admitted.
Set the auto-accept policy
The Auto-accept policy card is visible at every level but only editable at Auto-PR, so you can see what would apply before you switch. Its defaults are deliberately strict — see The auto-accept policy.
Set a spend budget
Fill in Limit (USD per period) and Period in the Autonomous spend budget card and press Save budget. A human clicking “start fix run” is, incidentally, a rate limiter; once that human is gone the budget is what bounds the bill. See The spend budget.
Read the decision log
The Autonomous decision log at the bottom of the tab records every admission, refusal, suspension, resume and cancellation with its reason and cost. It is the surface you check after the first day, not the PR list.
API-first
Every control has an API. Reads (GET) accept a nk_live_… agent key; writes require an admin
credential (an operator/company-admin key, or a signed-in user with the admin role). Autonomy has
its own admin-only authority, deliberately separate from the one that registers a VCS
installation: holding a repo credential is not the same privilege as authorising Neens to spend your
LLM budget and open pull requests unattended.
curl -s https://your-neens/api/autonomy/status \
-H "Authorization: Bearer nk_live_…"Returns the level, the suspension state and reason, the resolved policy, effectiveMinKappa, the
platformCeilings, live health signals, the current-period budget, inFlightRuns, runsToday,
lastSweepAt and the five most recent decisions — everything the tab renders, in one call.
The auto-accept policy
At Auto-PR — and only there — this policy decides whether a proposed remediation may become
accepted with no human. Every check must pass. A missing or unknown signal fails its check;
nothing passes quietly.
Three fields in this table are not Auto-PR-only. minVerifyKappa sets the κ floor that the
health gate uses to suspend the agent, and maxConcurrentRuns / maxRunsPerDay cap how many
unattended runs may start — all three are read at Auto-verify as well, before any admission
decision is reached. They stay editable at that level in the UI. The remaining six fields are
admission checks and apply at Auto-PR only.
| Field | UI label | Default | What it requires |
|---|---|---|---|
minClusterSize | Min cluster size | 5 | The remediation’s cluster has at least this many sessions of evidence |
minVerifyKappa | Min verifying κ (0–1) | 0.4 | Your κ floor; the effective floor is the higher of this and the platform floor |
minConfidence | Min remediation confidence (0–1) | 0.6 | The remediation’s own confidence score |
maxConcurrentRuns | Max concurrent autonomous runs | 1 | Autonomous runs in flight for this agent |
maxRunsPerDay | Max autonomous runs per day | 5 | Autonomous runs started today (UTC) |
requireContainedBlastRadius | Require a contained blast radius | true | Off, a broad change may also be accepted. protected and unknown never pass either way |
protectedGlobs | Protected paths (one glob per line) | (see below) | Any touched path matching one of these makes the change protected |
blockedCategories | Blocked failure categories (one per line) | auth, authentication, authorization, authn, authz, login, logout, password, credential, oauth, saml, sso, permission, rbac, encryption, pii, security, payments, billing, compliance, privacy | A failure in one of these subjects is protected |
allowedRemediationTypes | Allowed remediation types (one per line; blank = all) | (blank — all types) | Restrict auto-accept to specific remediation types |
The six checks are named in the decision record: cluster_size, confidence, verify_kappa,
blast_radius, remediation_type, status. Every one is evaluated even after an earlier one has
failed, so the log shows the whole card rather than making you fix thresholds one at a time.
status requires the remediation to still be proposed — anything else was already acted on.
Worked example — one admitted, one refused
Take an agent at Auto-PR with the default policy, an effective κ floor of 0.4, and a best
verifying judge at κ 0.61.
Admitted. A remediation for a grounding failure in the order-status agent: cluster of 41
sessions, confidence 0.82, type prompt_change, status proposed, fix artifact touching one file,
app/agents/order_status.py.
| Check | Result | Detail recorded |
|---|---|---|
cluster_size | ✅ | Cluster has 41 sessions; policy requires at least 5. |
confidence | ✅ | Confidence 0.82; policy requires at least 0.6. |
verify_kappa | ✅ | Best verifying judge κ is 0.61; the effective floor is 0.4. |
blast_radius | ✅ | Blast radius is ‘contained’; policy allows contained. 1 file would change, none of them protected. |
remediation_type | ✅ | Policy allows every remediation type. |
status | ✅ | Remediation is proposed. |
The remediation transitions to accepted (stamped as a scheduler action), a fix run is created with
origin autonomous, and an admitted decision is written carrying the full check array, the exact
policy that admitted it, the blast radius, κ and the cost estimate.
Refused. A remediation for a token-refresh failure: cluster of 63 sessions, confidence 0.91 —
stronger evidence than the first one — whose artifact touches services/auth/token_refresh.py and
services/api/session.py.
| Check | Result | Detail recorded |
|---|---|---|
cluster_size | ✅ | Cluster has 63 sessions; policy requires at least 5. |
confidence | ✅ | Confidence 0.91; policy requires at least 0.6. |
verify_kappa | ✅ | Best verifying judge κ is 0.61; the effective floor is 0.4. |
blast_radius | ❌ | Blast radius is ‘protected’; policy allows contained. Protected path services/auth/token_refresh.py matches **/auth/**. |
remediation_type | ✅ | Policy allows every remediation type. |
status | ✅ | Remediation is proposed. |
A refused decision is recorded with reason blast_radius, and the remediation stays proposed,
waiting for a human exactly as it does at propose_only. No amount of κ, confidence or cluster
size buys a way past this — see below.
A refusal is re-derived on every sweep tick, so an identical (remediation, reason) refusal is
written to the log at most once per 24 hours. A 15-minute cadence would otherwise record the
same refusal 96 times a day and bury the entries that matter.
Blast radius — what a fix is allowed to touch
Before any admission, Neens classifies how much of your system the remediation would move.
| Radius | When | Can auto-accept? |
|---|---|---|
contained | Up to 20 distinct touched files, none protected | Yes |
broad | More than 20 distinct touched files | Only with Require a contained blast radius unchecked |
protected | The failure’s category/label is a blocked category, or a touched path matches a protected glob | Never |
unknown | No touched paths could be derived for the remediation | Never |
The category check runs before the path check, because what a fix is about is known earlier and
more reliably than its file list. It matches the failure’s category and every label on the
remediation, on word boundaries and ignoring plurals — so a remediation labelled payment_flow or
categorised auth_failure is protected even if its artifact happens to name only docs/.
Word boundaries are why the default list spells out authentication, authorization, authn and
authz separately: the token auth matches auth_failure and authFailure, but not
authorization — that is one word, not two, and a substring rule would also catch author. If you
edit this list, add the spellings your cluster labels actually use rather than assuming a stem
covers them.
The default protected paths
These are the protectedGlobs defaults, verbatim. ** spans directories, and matching is
case-insensitive; a leading **/ also matches a top-level file, so **/*.tf matches main.tf as
well as infra/main.tf.
**/auth/**
**/auth*.py
**/access.py
**/access_control*
**/access_control*/**
**/login*
**/login*/**
**/logout*
**/password*
**/password*/**
**/credential*
**/credential*/**
**/oauth*
**/oauth*/**
**/saml*
**/sso*
**/*jwt*
**/rbac*
**/rbac*/**
**/permission*
**/permission*/**
**/payment*/**
**/billing/**
**/secrets/**
**/security/**
**/migrations/**
infra/**
**/*.tf
**/Dockerfile*
**/.github/workflows/**
**/iam/**
**/crypto/**The list is deliberately over-broad. A false protected costs one human click; a false contained
costs an unreviewed change to your auth path. Edit it per agent — but understand that an auth,
payments, billing, crypto, migration or CI-workflow change is refused regardless of κ, because κ
measures how well a judge agrees with your reviewers about quality, not whether an unreviewed
credential change is acceptable. Those are different questions, and no score answers the second one.
Why unknown fails instead of passing
An empty file list is the absence of evidence, never evidence of a small change. A remediation whose
artifact has not been generated yet, or whose diff could not be parsed, is exactly the case where
guessing “contained” opens an unreviewed PR. So unknown fails the blast_radius check and the
refusal is logged with that reason — the log says “we could not tell what this touches” rather than
going quiet.
The verifying-judge floor
A verifying judge must be human-anchored. The floor in force is the higher of your policy’s Min verifying κ (0–1) and the platform floor (default 0.4) — an agent policy can raise the bar, never lower it. The Auto-accept policy card shows the resulting Effective κ floor in force.
The κ that counts is the κ of a judge that will actually be allowed to verify the fix: judges the
fix was written against are excluded by the same anti-gaming rule the
eval-verified PR applies. If no
eligible judge clears the floor, the answer is None — not 0 — and autonomy suspends.
Fail safe, not fast
Before it considers a single candidate, each sweep asks whether the agent is still fit to act on its own. If it is not, autonomy suspends and says so — it never quietly drops to a weaker level, which would look like the product working while the guarantee it advertises is gone. The level you set is preserved through a suspension, so resuming restores exactly the autonomy you configured.
| Reason | What triggers it | Clears |
|---|---|---|
manual | A human pressed the kill switch (POST /autonomy/suspend) | A human resumes |
kappa_drift | An open judge-alignment drift alert: a verifying judge no longer agrees with human labels, so nothing it “verifies” can be trusted | A human resumes |
kappa_below_floor | No eligible verifying judge has a κ at or above the effective floor (including “no κ measured at all”) | A human resumes |
regression_failure | An autonomous run failed the accumulated regression set inside the look-back window (24 hours) | A human resumes |
budget_exhausted | The period spend limit is reached, or the next run’s cost cannot be priced | Clears itself when the budget period rolls over — or a human resumes sooner |
When several apply, the reported reason follows a fixed precedence:
kappa_drift → kappa_below_floor → regression_failure → budget_exhausted. That order runs from
“the measuring instrument is broken” to “the instrument says stop”: a drifting judge invalidates the
κ figure the next check would read, and an invalid κ invalidates the “verified” claim on the
regression runs after it. Reporting the shallowest symptom first would send you to fix the wrong
thing.
What you get told. A suspension raised by the fail-safe (any reason except manual) creates a
critical insight — “Autonomy suspended (…)” — which reaches the in-app feed and any Slack or
webhook sink you have configured under Insights & notifications. It also writes a
suspended row into the decision log and an autonomy.suspend entry into the
audit trail. A suspension you perform yourself is recorded in the decision log
and the audit trail, but raises no insight — you already know.
How to resume. Press Resume autonomy in the red banner at the top of the Autonomy tab, or
POST /autonomy/resume. Fix the underlying signal first: resuming with a drifted judge simply
suspends again on the next sweep. The one exception is budget_exhausted, which the sweep lifts by
itself at the period roll-over and records as its own decision and audit event — the money is back
and nothing about the agent’s health changed.
When one remediation keeps failing
A suspension stops the whole agent. This is the narrower case: one remediation the fix engine can never apply. After 3 consecutive failed autonomous runs, the sweep stops offering that remediation and moves on to the next candidate. Nothing else changes — the agent stays healthy, the loop keeps running, and every other remediation is still eligible.
It is a stop, not a slower retry, and that is deliberate. A remediation whose fix driver
produces no patch, whose artifact names no file, or whose target no longer exists will fail exactly
the same way on the hundredth attempt as on the third. Retrying it more slowly would keep spending
the daily run cap on a known-bad candidate and keep it ahead of the fixable remediations behind it —
candidates are selected highest-priority-first, so one unfixable priority: 99 remediation is
candidate #1 forever. A fix that cannot be applied is a signal a person should see, so the loop
hands it back to you instead of grinding on it.
Held back is not suspended. Autonomy is still on, the sweep still runs, and no one has to press Resume. A held-back remediation is offered again the moment a human touches it — there is nothing to clear and no flag to reset.
What counts toward the streak
The count walks that remediation’s fix runs newest-first and stops at the first thing that is not a failure. Only autonomous runs are read at all: a run you started by hand neither adds to the streak nor clears it.
| Run status | Effect on the streak |
|---|---|
failed | +1 |
cancelled | Neutral — skipped, and the walk continues past it. The loop halted that run (a suspension, or you lowering the level); that is not the remediation’s fault |
pr_opened, drafted, queued, applying, verifying | Stops the walk. The streak ends there — a run that got as far as a draft or a PR is not a consecutive failure |
| Any run that started before the remediation was last edited | Stops the walk. History from before a human last touched it is not held against it |
That last row is the reset. Editing, re-accepting or re-prioritising a remediation updates its timestamp, and only autonomous runs created at or after that moment are counted. The loop cannot clear its own streak this way: at Auto-PR the auto-accept write happens before the run it authorises, so it never post-dates the run it would need to hide.
If the streak cannot be read at all — a database error mid-sweep — the candidate is skipped for that tick and no decision row is written. “We could not check” must never be recorded as “this failed three times”.
How to clear it
Touch the remediation. Any change to it counts; you do not have to fix it first, though a remediation put back unchanged will simply fail three more times and be held back again.
Open it
Go to Remediations and select the remediation named on the Held back card.
Change something about it
Advance its Lifecycle (for example proposed → accepted), move its work state, or edit its
labels. Each of those is a write, and a write is the touch.
Wait for the next sweep
Within 15 minutes (the next sweep) it is a candidate again, with a streak of zero.
How to see it
The Held back card on Settings → Autonomy lists each parked remediation with its title, how many consecutive autonomous runs failed, when the last one failed, and that run’s failure text. It appears only while something is actually held back, and it is styled as a note rather than an alarm — nothing is suspended.
The decision trail records the hold as a refused decision with the reason
consecutive_failures. It is deduplicated for 24 hours per remediation, so a 15-minute sweep writes
one row a day, not ninety-six:
# exactly what the loop is holding back, newest first
curl -s "https://your-neens/api/autonomy/decisions?reason=consecutive_failures&limit=50" \
-H "Authorization: Bearer nk_live_…"
# or scope it to the one remediation
curl -s "https://your-neens/api/autonomy/decisions?decision=refused&remediationId=rem_abc" \
-H "Authorization: Bearer nk_live_…"The endpoint filters on decision, remediationId and reason, in any combination. Filtering by
decision=refused alone is a wider question — it returns every refusal the auto-accept policy has
ever made, most of which are ordinary “this one did not meet the bar” decisions rather than holds.
An insight — autonomy_remediation_quarantined — is raised in the Insights
feed, deduplicated per remediation, so it reaches whatever Slack or webhook sink you have configured
without repeating every sweep.
Worked example
An agent at Auto-PR with maxRunsPerDay: 5, a spend budget of $40/month, and one remediation
rem_abc (priority: 99) whose artifact the fix driver cannot apply. Each run is estimated at
$2.00 and fails in about two seconds, before the driver produces anything.
Without the quarantine. Every 15-minute tick re-admits rem_abc — the run failed, and a failed
run does not block re-admission. Five runs a day, five reservations settled at $2.00 each: $10/day
of spend against a run that never called a model, and none of the five daily slots ever reaches a
remediation that could have been fixed. Four days in, the $40 month budget is full and the agent
suspends with budget_exhausted — for money nobody spent.
With the defaults. Runs 1, 2 and 3 fail the same way, and because none of them created a
verification run each reservation is released rather than billed: the period total is still
$0.00. On the fourth sweep the streak is 3, so rem_abc is not a candidate. One refused decision
with consecutive_failures is written, one autonomy_remediation_quarantined insight is raised,
and the sweep admits the next remediation down the priority list. The daily cap of five now belongs
to the remediations that can actually be fixed. When someone repairs the artifact and re-accepts
rem_abc, the next sweep sees no autonomous failures since that edit and offers it again.
The kill switch, mid-flight
A fix run takes minutes of work. Switching autonomy off has to affect work already in flight, not just work not yet started — otherwise “I turned it off” still opens a PR ten minutes later.
Three actions trip the kill switch: suspending the agent, lowering the level out of Auto-verify/Auto-PR, and any automatic suspension from the table above. Each one:
- Cancels every in-flight fix run whose origin is
autonomous, writing statuscancelledwith the reason on the run. A run that already reached a terminal state (pr_opened,drafted,failed,cancelled) is left exactly as it is — nothing already produced is rewritten. - Releases the run’s spend reservation, so cancelled work does not hold budget.
- Records a
cancelleddecision per run, so the log explains why the run stopped.
The worker enforces the same thing from its side: an autonomous run re-reads its agent’s autonomy state at the top of every attempt and immediately before it would open the pull request. An agent whose autonomy was switched off never wakes up to a PR nobody sanctioned.
Runs a human started are never touched. Switching the unattended loop off is not an instruction
to cancel a person’s work, so manual-origin runs keep going.
When a run’s worker dies
A fix run is executed by a worker process, and a process can disappear: an out-of-memory kill, a pod eviction, a lost node. When that happens between the moment the worker picks the run up and the moment it reaches a result, the run is left mid-flight with nothing alive to finish it. Its own error handling cannot help — that code died with the process.
Left alone, one such run does three kinds of damage, in increasing order of how long it takes you to notice:
- The run shows as in progress in the UI. Forever.
- It holds its spend reservation, so the period’s budget is permanently smaller.
- It counts against the concurrent-run cap. At the default
maxConcurrentRunsof 1, that means every later sweep hits the cap and starts nothing — the unattended loop is stopped, indefinitely.
Neens handles all three automatically. This section is about how to recognise it and what you will see.
”It says 1 in flight and nothing has happened for days”
That is the symptom, and it used to be genuinely ambiguous: a run started thirty seconds ago and a run whose worker died last Tuesday both read as In-flight autonomous runs: 1.
Settings → Autonomy now puts an age next to the count:
In-flight autonomous runs: 1 · oldest seen 6 hours ago
Autonomous runs today: 0
Last sweep: 12 minutes agoRead it like this:
| What you see | What it means |
|---|---|
| A count with a recent “oldest seen” (minutes) | Normal. A fix run takes minutes; verification is the slow part |
| A count with an “oldest seen” of hours or days, in warning colour, plus the panel A fix run’s worker has stopped checking in | The worker is gone. The reaper will cancel the run on its next pass and give the slot back |
| Autonomous runs today: 0 with a non-zero in-flight count | Nothing new is being started. Check the decision log for a concurrency_cap refusal |
| Last sweep stuck far in the past | Different problem — the sweep itself is not running. Check that the scheduler process is up |
The same numbers are on GET /autonomy/status, as inFlightRuns, staleInFlightRuns,
oldestInFlightRunId, oldestInFlightRunLastSeenAt and oldestInFlightRunAgeSeconds.
A stale run does not make health.healthy false and does not suspend the agent. Health is
about your fix quality gates (κ, drift, regressions, budget) and every one of its reasons needs a
human to resume. A stale run needs no human — it heals on the next pass, usually within minutes.
What the reaper does
A background job — the stale-run reaper — looks for in-flight fix runs whose worker has stopped reporting, and cancels them. It runs every 5 minutes, independently of whether autonomy is enabled, and it covers runs of every origin: a run you started by hand is just as dead when its worker vanishes; it simply is not holding the autonomous cap. The autonomy sweep also runs the same check immediately before it counts the caps, so the cap heals on the very path that enforces it.
While a run is executing, its worker stamps a liveness heartbeat every 30 seconds. A run is stale when:
| Situation | Stale after |
|---|---|
| The run has a heartbeat and it has gone quiet | 15 minutes |
| The run has no heartbeat at all — it was never picked up, or it was started before this release | 180 minutes |
The two bounds are deliberately far apart, because the two signals mean different things. A missed heartbeat is positive evidence of death: nothing but a live worker writes that timestamp. General silence is only an absence of evidence — a healthy run spends most of its wall time inside verification, replaying your golden dataset k times without writing anything — so it gets a much longer rope.
What a reaped run looks like
The run
Status cancelled, and its error says what was observed:
the worker running this fix run stopped reporting: no sign of life since 2026-08-01T04:12:55+00:00 (372 minute(s) silent). Neens cancelled the run and released its budget reservation. Nothing is known to be wrong with the fix itself — the process executing it disappeared (an OOM kill, a pod eviction or a lost node). Start a new run when you are ready.
The budget
The run’s reservation is released. It disappears from the period’s reserved total and the headroom comes back. A settled cost is never unwound — only a reservation that was never spent.
The decision log
For an autonomous run, one cancelled row with reason worker_lost, rendered as The run’s worker
stopped checking in, so the run was cancelled, carrying the same sentence as the run’s error.
The insight feed
A fix_failed insight, Fix run abandoned (worker lost) — the feed you already read, rather than
a new surface nobody has learned to look at.
The next sweep
The slot is free, so the loop starts a run again on its next tick. Nothing needs a human.
Why cancelled and not failed: because the fix did not fail — the process running it vanished,
and calling that a failed fix would be a claim about your patch that Neens has no evidence for.
(There is a second, quieter reason: cancelled is the one status a straggler worker can never
overwrite, so the reaper’s verdict is final by construction.)
Nothing about a reaped run marks the remediation as bad. Start a new run whenever you like — the remediation, its proof gate and its regression set are untouched.
When the loop is at a cap, it now says so
Both cap skips write a refused row into the decision log instead of returning silently. They are
deduped over 24 hours, so a 15-minute sweep records one row per reason per day rather than 96.
| Reason | What it means | What to do |
|---|---|---|
concurrency_cap | Enough autonomous runs are already in flight. The detail names the counts, the effective cap, the oldest in-flight run’s id and how long since it was last seen, and whether any of them are stale | If the oldest run is stale, nothing — the reaper takes it and the loop resumes. If the runs are genuinely working, this is the cap doing its job; raise Max concurrent runs if you want more parallelism |
daily_cap | This agent has already started its allowance of autonomous runs today (UTC) | Usually nothing — the count resets at 00:00 UTC. Raise Max autonomous runs per day if the loop should get further each day. Both caps are still bounded by the platform ceilings |
Because the row is deduped, its detail is the state at the first refusal of the window, not a live
figure. The live one is always on Settings → Autonomy / GET /autonomy/status.
Worked example
An agent at Auto-verify with maxConcurrentRuns: 1. Times are relative.
00:00 — a run is admitted
The sweep finds an accepted remediation, estimates the run at $0.18, reserves it, creates the run and
records an admitted decision. In-flight: 1 · oldest seen just now.
00:03 — the worker is OOM-killed
The run is mid-verification. Its last heartbeat was seconds ago; the row still says it is running.
00:03–00:18 — inside the stale bound
The run is silent but not yet stale — this is indistinguishable from a slow verification, on purpose.
In-flight: 1 · oldest seen 15 minutes ago. The sweep at 00:15 finds the cap consumed and records
one refused / concurrency_cap row naming this run and its age.
00:18 — it crosses the bound
The Autonomy tab turns the count to its warning colour and shows A fix run’s worker has stopped
checking in — the oldest is fxr_…, last seen 18 minutes ago.
≤00:23 — the reaper takes it
Status cancelled with the “no sign of life since…” error. The $0.18 reservation is released. A
cancelled / worker_lost decision and a Fix run abandoned (worker lost) insight are recorded.
≤00:38 — the loop resumes
The next sweep sees zero in-flight runs, and admits the next eligible candidate. In-flight: 1 · oldest seen just now. Total unattended downtime: under 40 minutes, with a written explanation for every minute of it.
Before this existed, step 5 never happened, and steps 3 and 6 lasted until somebody noticed by hand.
The spend budget
The Autonomous spend budget card caps what the fix engine may spend on this agent per period. Set Limit (USD per period), a Period (Per day, Per week (ISO), Per month) and tick Enforce this budget.
How the estimate is computed
Before an autonomous run starts, Neens prices it and reserves that amount against the period:
verification runs = passK + (1 if the accumulated regression set is non-empty else 0)
total items = passK × (proof-gate dataset items) + (regression set items)
input tokens = total items × 1200 (assumed per item)
output tokens = total items × 512 (assumed per item)The item counts are read, never assumed — the proof-gate dataset’s frozen golden version and the
real regression set. With passK at its default of 3, a 14-item gate dataset and a 210-item
regression set: 4 verification runs, 3 × 14 + 210 = 252 items, 302,400 input and 129,024 output
tokens, priced at the rate in force for the model on the agent’s default LLM connection.
Prices resolve through your tenant’s price table — the shipped catalogue plus any rate you set yourself under Settings → Model pricing — so a self-hosted or open-weight model you have priced yourself is priced here too, and a negotiated rate beats the list price. See Cost & model pricing.
The honesty rule: unpriced is never $0
If the model has no rate, the estimate is not zero — it is absent. The card shows
Cost could not be priced and the decision log shows — could not be priced in the Cost
column, never $0.00.
This matters more here than anywhere else in the product. A free run fits inside every budget, so
treating an unpriced model as $0 would turn it into an unlimited autonomous spend allowance — the
exact inversion of what a budget is for. So the verdict becomes Cost unknowable, and an
unknowable cost refuses the autonomous run and suspends the agent with budget_exhausted.
| Verdict | Condition | Effect |
|---|---|---|
Within budget (ok) | No limit configured, or spent + reserved + estimate ≤ limit | Autonomous runs may spend |
Budget exhausted (exhausted) | spent + reserved + estimate > limit | Suspends with budget_exhausted |
Cost unknowable (unknown) | The estimate cannot be priced, or any spend already in the period could not be priced | Suspends with budget_exhausted; the detail says the cost is unknowable rather than over the limit |
A manual run is still allowed in every one of these states. What is refused is spending money nobody
can bound without a person looking at it. To clear an unknown, add the missing rate under
Settings → Model pricing — the card links straight there, and the rate you set is the one the
next estimate uses.
These are estimates, not metered actuals
Neens does not measure what an individual fix run cost. Set your limit accordingly: budget enforcement is arithmetic over estimates, and the ledger says so rather than dressing them up.
- Before a run, its estimated cost is reserved against the period.
- When the run reaches a terminal state having run at least one verification run, that
reservation is settled at the same admission estimate — recorded as priced when the estimate
was priceable, and marked in the ledger entry as an estimate rather than a measurement
(
source: "admission_estimate",measured: false). A run that never verified anything is released instead; see below. - A ledger row is unpriced (
priced: 0) only when the cost was genuinely unpriceable — an unpriced model — never merely because it was not metered. So a successful run against a priced model does not turn the periodpartial, and an agent with a healthy budget is not suspended by its own first success. - On the decision row,
estimatedUsdMicrosandcostPartialcarry the admission figure, whileactualUsdMicrosstays null — a field named “actual” must not be filled with an estimate.
remainingMicros is not clamped at zero, so an already-overspent period reports a negative number
rather than a reassuring 0, and the usage bar is only drawn when the total is trustworthy.
A run that verified nothing is released, not billed
The admission estimate prices verification runs — passK proof-gate runs plus a regression run —
so it is only meaningful once at least one of them exists. A run that reaches a terminal state
without ever creating a verification run therefore has its reservation released, not settled:
the ledger records it as kind: released and the period total is unchanged.
That covers every way an autonomous run can die before it costs anything: the remediation was
deleted between admission and execution, the configured fix driver could not be built, the driver
produced no patch, or the run drafted a change with no verification target. All of them used to
settle the full estimate — a run that failed in two seconds having called no model still moved the
budget, and enough of them suspended an agent on budget_exhausted for spend that never happened.
A run that got partway through still settles the whole estimate. If one verification run of three completed and the run then failed, the full admission estimate is booked. Neens records no measured per-run cost, so there is no honest way to bill a fraction of it — and for a control whose job is bounding unattended spend, over-billing is the safe direction. The alternative is a budget that silently under-counts real money.
You can tell the two apart in the ledger (Autonomous spend budget → the current period’s rows,
or GET /spend-budgets/fix_engine/usage): a reserved row followed by settled is spend that
counts, and a reserved row followed by released is spend that was authorised and then given
back.
The audit trail
Every autonomous action is reconstructable after the fact, in two places.
The decision log (Settings → Autonomy, or GET /autonomy/decisions) holds one row per
decision:
| Decision | Written when |
|---|---|
admitted | A candidate cleared everything and a fix run was created |
refused | A candidate failed the policy, had no proof-gate dataset, or is held back after too many consecutive failures (reason consecutive_failures) — or the sweep stopped at a cap before looking at any candidate (concurrency_cap, daily_cap). Deduped over 24 h per remediation and reason |
suspended | The fail-safe fired, or a human hit the kill switch |
resumed | A human resumed, or a budget period rolled over |
cancelled | An in-flight autonomous run was cancelled — by the kill switch, or by the stale-run reaper with reason worker_lost |
Each row carries the decision, level, remediation and cluster, the fix run, a short machine reason
and a human sentence, the full checks array, the exact policy snapshot that decided it, the
blast radius and what matched, the verifying κ, and the estimated cost with a flag for whether
it could be priced. actualUsdMicros stays null: Neens measures no per-run cost and will not present
an estimate as one. Filter with ?decision=refused, ?remediationId=rem_… or ?reason=…
(?reason=consecutive_failures is the held-back set).
The audit trail records the mutations alongside every other change in the
tenant: autonomy.update (with the before/after level and policy, and any runs cancelled),
autonomy.suspend, autonomy.resume, autonomy.start_run (per autonomous run, with the blast
radius, κ, estimate and decision id) and spend_budget.update.
A fix run started by autonomy is also marked at the run itself: origin is autonomous and
autonomyDecisionId points back at the decision that authorised it.
How it works
A scheduled sweep runs every 15 minutes across
every agent that has opted in to a non-off level. An agent that has never opened the tab costs
one indexed read per tick — no rows are created for it. Per agent, in this order:
- Lift a rolled-over budget suspension — and only a budget suspension. A κ drift does not heal at midnight.
- Stop if suspended. Nothing is recorded; the suspension was already surfaced when it fired.
- Gather health signals — open judge-drift alerts, the best eligible verifying κ, recent
autonomous regression failures, and the period budget. These are the same four numbers
GET /autonomy/statusshows you. - Run the fail-safe gate. If it says stop: suspend, notify, cancel in-flight autonomous runs, and return.
- Reap stale in-flight runs before counting them, so a run whose worker died cannot consume the concurrency cap. See When a run’s worker dies.
- Apply the caps. The effective cap is the lower of your policy value and the platform
ceiling (2 concurrent runs; 20 runs per day). Only autonomous runs count toward them — a
human’s three manual runs do not consume the unattended loop’s budget, and the loop never makes
room by cancelling someone’s work. Either cap being hit records a deduped
refuseddecision, so a tick that did nothing still says why. - Select candidates, highest priority first.
auto_verifyconsidersacceptedremediations only;auto_pradditionally considersproposedones, each of which must still clear the whole auto-accept policy. A remediation that already has a fix runqueued,applying,verifying,pr_openedordraftedis skipped, which is what makes the sweep safe to re-run. A remediation with 3 consecutive failed autonomous runs since a human last touched it is skipped too — see When one remediation keeps failing. - Per admitted candidate: estimate the cost → re-check the budget with that estimate included → reserve → create the run → record the decision → commit → enqueue. “We are under the limit” is not the same claim as “this run fits under the limit”, so the second check is not redundant. One malformed candidate never aborts the sweep, and a failure after the reservation releases it.
Reference
Routes
| Method + path | Who | What |
|---|---|---|
GET /autonomy/status | read | Level, suspension, policy, effective κ floor, platform ceilings, health, budget, in-flight and today’s run counts (with the oldest in-flight run’s id, age and last-seen time, plus a stale count), last activity, five recent decisions |
GET /autonomy/settings | read | Level, suspension state and the resolved policy |
PATCH /autonomy/settings | admin | Set level and/or merge a policy patch. Lowering out of the autonomous levels cancels in-flight autonomous runs |
POST /autonomy/suspend | admin | The human kill switch; optional reason and detail |
POST /autonomy/resume | admin | Clear a suspension; 409 when not suspended |
GET /autonomy/decisions | read | The decision trail; limit, offset, decision, remediationId, reason |
GET /spend-budgets | read | Every feature’s budget for the agent |
GET /spend-budgets/{feature} | read | One feature’s limit, period and enabled flag |
PUT /spend-budgets/{feature} | admin | Set limitUsd (null = unlimited), period, enabled |
GET /spend-budgets/{feature}/usage | read | The current period’s spend, verdict and ledger rows |
{feature} is fix_engine today; model_sweep is accepted as a name but nothing bills against it
yet. Reads accept a nk_live_… agent key; writes need an admin credential.
Troubleshooting
- The Autonomy tab is missing. You are not an admin — the tab is admin-only.
- The level is
auto_prbut nothing is ever admitted. Open the decision log and read therefusedrows — they name the failing check. The most common areblast_radius(unknown: the remediation has no derivable file list) andverify_kappa(no judge is anchored). Ano_proof_gaterefusal means the remediation has no eval gate to verify a fix against. - Autonomy suspended with
kappa_below_flooron a brand-new agent. No judge has a measured κ yet, which reads as “nothing is anchored” — not as zero. Label gold items so alignment is measured (see Annotations & review), then resume. - Autonomy suspended with
budget_exhaustedand the detail says the cost is unknowable. The model on the agent’s default LLM connection has no rate. Add it under Settings → Model pricing (Cost & model pricing), then resume. - The budget shows “Cost could not be priced” even though runs are finishing. That state means a
model genuinely has no rate — not that a run went unmeasured. Find the unpriced model and give it a
rate in Settings → Model pricing; the estimate then prices, the period stops being partial, and
the verdict leaves
unknown. A priced run settling does not make the period partial. - One remediation stopped being picked up, but nothing is suspended. It is held back after 3 consecutive failed autonomous runs. Look at the Held back card on the Autonomy tab for the last failure text, fix whatever it names, and touch the remediation — any edit puts it back in the queue. See When one remediation keeps failing.
- A remediation is held back and I do not think it should be. Read the last failure report on the card. If the failures were caused by something agent-wide that you have since fixed, just re-accept the remediation: the streak only counts autonomous runs started after your edit.
- The period total looks lower than my provider invoice. It is a sum of pre-run estimates, not metered spend — Neens records no per-run actual. Treat the limit as a governor on how much unattended work may be authorised, and reconcile real spend with your provider.
- A run vanished into
cancelledwith “autonomy cancelled this run”. Someone lowered the level or suspended the agent while it was flying. That is the kill switch working; the decision log names the reason. In-flight autonomous runs: 1for hours and nothing new ever starts. The run’s worker died and is holding the concurrency cap. The count shows the oldest run’s age beside it, and a warning panel appears once it is stale; the reaper cancels it within a few minutes and the loop resumes on its own. See When a run’s worker dies.- A run was cancelled with “the worker running this fix run stopped reporting”. The reaper took it. Nothing is known to be wrong with the fix — start a new run. If this happens repeatedly, look at worker memory limits and evictions rather than at the remediation.
- The decision log shows a
concurrency_capordaily_caprefusal. The loop stopped at a cap before considering any candidate. Read the detail: aconcurrency_caprow naming an in-flight run that was last seen hours ago is a dead worker, not a busy loop. - I resumed and it suspended again within 15 minutes. The underlying signal is unchanged. Resume clears the flag, not the cause.
Related
- Eval-verified PR — the run autonomy starts, and the proof it carries
- Remediations — where candidates come from, and what
proposed/acceptedmean - Eval gates — the proof gate a candidate needs before it can be admitted
- Annotations & review — how judge↔human alignment (κ) is measured
- Fix outcomes — what happened after the PR merged
- Insights & notifications — where a suspension alert lands