FAQ

Common questions and troubleshooting for Neens. If something isn’t covered here, ask the in-app Assistant or your workspace admin.

Getting started

How do I send my first trace?

Create an API key in Settings → API keys, then point your OpenTelemetry exporter at Neens or POST a trace to /ingest/raw. See Getting started and Send traces.

Which names do I have to spell exactly?

The ones a typo makes silent rather than loud: environment variables (NEENS_*), the pre-prod correlation attributes (neens.eval_run_id / neens.dataset_item_id / neens.version_label), the request headers (X-Neens-Project-Id / X-Neens-Company-Id), and the API key prefixes (nk_live_ for an agent key). Everything else — trace attributes, endpoints, field names — is either standard OpenTelemetry or discoverable from the UI and the API responses.

Do I need to change my agent’s code to use Neens?

If your agent already emits OpenTelemetry traces, no — point the OTLP exporter at Neens and add your API key as a bearer token. Otherwise you can POST traces in the Neens-native raw JSON format. See Send traces.

What’s the difference between a trace and a session?

A trace is one end-to-end run of your agent. A session groups related traces (a multi-turn conversation) by their shared conversation id. Traces shows individual runs; Sessions shows the conversation-level rollup. See Traces & sessions.

Ingestion

My traces aren’t showing up

Work through these:

Wrong key or agent. A trace lands in the agent its API key belongs to. Make sure you’re viewing that agent (use the agent switcher in the sidebar).
Auth header. The key must be sent as Authorization: Bearer nk_live_…. A 401 means the credential was rejected — see the next question.
Give it a few seconds. Ingestion is asynchronous — Neens returns 202 Accepted immediately and the trace appears shortly after, once the background worker has processed it. Under load this lag can grow. To confirm a single trace synchronously while testing, add ?sync=true to the ingest URL and you’ll get the stored session id back.
Payload shape. For raw JSON, session.id is required. A malformed or unparseable body returns 422 with a message saying what’s wrong.
Size limits. A single trace payload over 1 MiB (or a batch over 8 MiB) is rejected with 413.
Backpressure. A 429 with a Retry-After header means Neens is deliberately throttling you — see below.

Why am I getting a 401 on ingest?

Neens never silently accepts a bad credential. If you supply a key and it doesn’t resolve, the request is always rejected with 401 rather than being misfiled:

Invalid or revoked API key. — the nk_live_… key you sent doesn’t exist, was mistyped/truncated, or has been revoked. Revocation takes effect immediately. Create a new key in Settings → API keys and make sure you copy it exactly.
Unrecognized API credential. — you sent a bearer token that isn’t a Neens key at all (for example, a provider API key by mistake).
Ingest requires an API key. Provide 'Authorization: Bearer nk_live_…'. — you sent no credential, and this deployment requires ingest authentication. Mint a key in Settings → API keys and send it as Authorization: Bearer nk_live_… — see API keys.

Why am I getting 429 responses on ingest?

Your agent’s ingest queue is backed up and Neens is applying backpressure instead of accepting work it can’t process yet. The response includes a Retry-After header (5 seconds by default) — honor it and retry with backoff; nothing is lost as long as you retry.

What formats does Neens accept?

OpenTelemetry (OTLP, protobuf or JSON) at /v1/traces, OpenInference at /ingest/openinference, and Neens-native raw JSON at /ingest/raw (or /ingest/batch for many at once). All accept gzip-compressed bodies. See Send traces.

Is there a size limit?

Yes — a single trace payload can be up to 1 MiB and a batch up to 8 MiB (measured after decompression). Larger payloads are rejected with 413.

Can one API key write to multiple agents?

No. A key is bound to a single agent, which is why you don’t send an agent ID with your traces — the key determines the destination and can’t be used to write elsewhere.

Why do my Sessions and Traces counts differ?

They count different things. Traces counts individual runs; Sessions counts conversations. Traces that share a conversation id collapse into one session, and a trace without one is its own single-trace session — so the session count is always less than or equal to the trace count. If everything shows as single-trace sessions, your instrumentation isn’t emitting a conversation id (gen_ai.conversation.id, session.id, conversation.id, or thread.id). See Traces & sessions.

AI features & LLM connections

AI features aren’t working / my clusters are unlabeled

Neens ships with no built-in model access — every AI feature (judge scoring, cluster labels, insight summaries, topic classification, enrichments, remediation drafts, the assistant) runs on an LLM connection your workspace configures. If none is configured, or none is visible to your agent (connections can be scoped to specific orgs/agents), those features degrade gracefully rather than erroring:

traces still ingest and display normally,
failure clusters are still detected but stay unlabeled,
judge runs fail with a clear error instead of producing scores,
the assistant can’t answer.

Fix: an admin adds a connection in Settings → LLM providers (Anthropic, OpenAI or any OpenAI-compatible endpoint including local Ollama, or AWS Bedrock) and, if it’s scoped, makes sure your agent is included. Use the connection test — it makes one real model call — to confirm the credential works. See Getting started.

Can the assistant change my data?

Only with your explicit approval, and only within your own permissions. The in-app Assistant can curate datasets, record ground-truth labels, create and deploy judges, start eval runs, and move suggested fixes along — but it always shows you the change first and runs it only when you press Approve. The change is then made on your own credentials, so it can never do something your role doesn’t already allow, and it is attributed to you — recorded against your name, and shown as having come from the Assistant wherever that resource keeps an audit trail. Nothing deletes: there is no delete tool. See Changes it can make.

Scoring & evaluation

Are my traces scored automatically?

Yes, once an LLM connection is configured. New agents come with a Primary Score composite judge that continuously scores a sample of incoming traces. You can add more judges or run them on demand. See Continuous evaluation and Judges.

Why are only some of my traces scored?

Continuous (on-ingest) scoring runs on a sample of traffic to control cost — 5% by default for the Primary Score, plus a per-day budget per judge deployment. To score a specific set, run a judge on demand against the traces or a dataset you choose — manual runs score every selected target, with no sampling. See Continuous evaluation.

I ran a judge but no scores appeared

Check the run itself before the scores:

Open the run in Activity. Every judge run appears in the Activity feed with its status and an error summary. A run that errored on every item produces no scores. See Activity.
Understand the status. A run’s status reflects its per-item success rate: completed (at least 90% of attempted items scored successfully), completed_with_failures (in between), failed (under 10% succeeded).
Per-item errors mean per-item gaps. When the judge’s model call fails on one item (after up to 3 automatic retries for transient errors), that item is marked failed and simply has no score — the rest of the run continues.
Common root causes: no LLM connection visible to the agent (above), a revoked or exhausted provider credential, or provider rate limits — the Activity error summary tells you which.

How do I trust a judge’s scores?

Use judge alignment: Neens compares judge scores against human labels, shows how closely they converge, and lets you drill into disagreements and relabel. See Annotations & review.

What’s the difference between a judge and an enrichment?

A judge produces a score (a quality measurement). An enrichment adds structured metadata (a category, sentiment, extracted fields) to make traces easier to filter and segment. See Judges and Enrichments.

Pre-prod evaluations

How do I test a new agent version before shipping it?

Run a pre-prod evaluation: replay a golden dataset against a candidate version, score the results with your existing judges, and compare against a baseline (a previous run or a production time window). Regressions gate the release. You can run your agent yourself and let Neens correlate the traces, or register your agent’s HTTP endpoint and let Neens call it for every golden prompt. See Pre-prod evaluations.

Neens refuses to call my agent endpoint

That’s the SSRF guard. When Neens calls your agent it makes outbound HTTP calls to the endpoint you registered, so by default it refuses any URL that resolves to a private, loopback, or link-local address — the item fails with an error like agent endpoint url is not allowed: host '…' resolves to a disallowed (private/loopback/link-local) address.

If your agent genuinely lives on an internal address (a Docker service, a VPC host, localhost in development), an operator can allowlist it. Only expose endpoints you control. Also note each endpoint call is capped at 60 seconds — a hung endpoint fails that one item honestly rather than fabricating output.

Do pre-prod evals affect my production metrics?

They never slow production down — pre-prod runs execute on an isolated worker fleet. They are not automatically excluded from your dashboards, though: pre-prod scores carry source: preprod, which makes them their own score type and easy to filter, but a score-grain widget counts every source unless you say otherwise. Slice or filter by Score source (score_source) when you want production-only numbers. See Pre-prod evaluations.

Diagnosing failures

How does Neens decide what’s a failure?

By default it treats a trace as failing when its primary score falls below 0.5 — so it catches “silent” failures, not just runs that errored. Admins can change the selection criteria per agent (score-based with a chosen metric and threshold, all traces, or errors only). See Clustering.

What’s the difference between a failure mode and an issue?

A failure mode is a category in your failure taxonomy — Neens ships platform defaults and you can add your own. An issue is a failure mode being actively tracked as work, with a lifecycle (open → acknowledged → investigating → mitigating → resolved, plus mute). See Issues & failure modes.

Why does a fix say “this is a downstream service issue, not your agent”?

Because the evidence says so. When Neens generates a remediation it classifies where the failure lives from the traces — the HTTP status codes in the errors, the error classes, the span kinds. If your agent’s tool is getting 5xx/503/timeouts from its own backend (or from a service it calls), that’s a service failure, not an agent defect — so Neens generates an advisory that names the owning component and recommends a resilience fix (backoff, circuit breaker, idempotency key) instead of a pointless prompt edit. See Where the failure lives.

Why is my remediation held as “Needs grounding” and not in the backlog?

A remediation only enters the actionable backlog if it carries a concrete change. If the proposal came back empty or templated (e.g. generic “clarify the agent’s instructions” filler), or there wasn’t enough evidence to classify the failure, Neens parks it as needs grounding rather than pass off a placeholder as work. It’s not lost — switch to the Needs grounding view (or add include_ungrounded=true to the API call) to see and work it. See Needs grounding.

Why can’t I mark a fix as applied?

Applying (or verifying) an agent-fixable remediation requires a bound proof — a verification run or a proof-eval gate — so “we shipped it” and “we proved it works” stay distinct claims. Simulate the fix or run an eval-verified PR to bind one; if you’ve verified it another way you can override with a recorded reason. See Applying a fix requires a bound proof.

Emails & notifications

Emails aren’t arriving

Neens sends transactional email for password resets, member invites, first-admin activation, and the optional daily digest. If none of them arrive:

Check with your admin that an email provider is configured. If none is, no email is sent.
No provider ⇒ emails are logged, not sent. With no transport configured, Neens never silently drops a message — it writes the full email, including the activation or reset link, to the server logs. An admin can copy the link from the logs to unblock you; invite flows also surface the raw link in the UI as a fallback.
Expired link? Password-reset links are valid for 60 minutes; invitation/activation links for 14 days. Request a fresh one.

Access & administration

Who can do what?

Admins manage the workspace — team, agents, LLM connections, and keys. Members work within the orgs and agents they’ve been assigned; viewers are read-only. See the Administration section of these docs.

How do teammates get access?

An admin invites them by email from Settings → Members; the invite is a one-time activation link (valid 14 days) where they set a password. New members can be assigned to specific orgs on invite.

I lost my API key — can I recover it?

No. Neens stores only a hash of each key, so the raw value is shown only once at creation. Revoke the old key and create a new one — revocation takes effect immediately.

What is a persona? Does it limit what I can do?

A persona is a lens: it tailors which navigation items are emphasized, your landing page, and defaults to how you work (Executive, Developer, Product manager, …). It never grants or removes access — everything stays reachable, and you can switch or reset your lens any time.

Data & privacy

Is my data isolated from other companies?

Yes. Every trace is scoped to your agent and company, and each company’s data is kept physically separate. You only see data in the agents and orgs you have access to.

How long is my data kept, and can I delete a specific person?

An admin sets a retention window (a company-wide default plus optional per-agent overrides; 0 = keep forever), and Neens purges data older than the effective window nightly. To satisfy a GDPR Art. 17 request, an admin can run an irreversible subject erasure that hard-deletes one subject across every store, with a deletion manifest and audit trail. See Data retention and Erasure.

How is spend calculated?

From the token usage on your model spans — Neens reads input/output tokens from standard attributes and prices them per model. See Metrics.

Why does my spend figure say partial, and how do I fix it?

Some traces in that window ran on a model with no configured rate. Neens has no fallback rate, so those tokens are excluded from the total rather than priced at a guess — meaning the number shown is a floor, and real spend is higher. Hover the badge’s ⓘ to see which models are unpriced (click it to pin the popover open so you can follow the link inside; Escape or a click outside closes it), then have an admin set a rate for each in Settings → Model pricing. Rates are per 1M tokens, and 0 is a valid rate for a model you self-host. Full walkthrough: Fix a spend figure that says partial.

Can I prove a cheaper model still hits our quality bar?

Yes. Every score records which model produced the answer it graded, so eval_pass_rate slices by model — and by agent × model, since a cheaper model is often fine for one agent and not another. A trace that used two models lands in an explicit Mixed bucket and a score with no derivable model in an Unknown one; neither is folded into a real model’s rate. See Model comparison, and Cost & model pricing for the cost half.

Erasure