LLM connections

Neens reads no global LLM API key. Every LLM-powered feature — judge scoring, insight summaries, the assistant, topic and issue classification, enrichments, remediation generation, cluster labeling — resolves an LLM connection you configure per workspace, org, or agent. Your credentials stay yours: they are encrypted at rest and never echoed back by any API.

At a glance

API formats

The API format you choose describes the API protocol the endpoint speaks, not the model vendor. Most hosted and self-hosted model services speak the OpenAI chat-completions API, so for anything that isn’t Anthropic, Google Gemini, or one of the named formats below, pick OpenAI-compatible and point it at the endpoint.

To point Neens at your own agent’s HTTP endpoint for pre-prod evaluations — where Neens replays each golden prompt against your agent — use the separate Add agent endpoint button on the same page, not the API-format dropdown. See Testing an agent endpoint below.

Create a connection

Worked examples

DeepSeek (an OpenAI-compatible vendor)

DeepSeek, Together, Fireworks, vLLM, LM Studio, and most hosted or self-hosted gateways speak the OpenAI chat-completions API. You pick OpenAI-compatible — the API format, not a vendor name — and tell Neens which endpoint to call.

Google Gemini

Testing an agent endpoint

The Test action for an agent_http connection calls your endpoint exactly the way the pre-prod worker will, with a trivial ping prompt. This probe is SSRF-guarded: a base URL whose host resolves to a private, loopback, or link-local address is refused unless an operator has explicitly allowlisted that host. Redirects are never followed. That allowlist is what lets a deliberately internal endpoint — a Docker service, a VPC host, localhost — be probed and called on purpose.

Credentials and secrets

• Credentials are write-only: you can set or replace one, but no read endpoint ever returns it. The UI shows only whether a usable credential is present.
• Stored credentials are encrypted at rest using a symmetric key. If the deployment has no encryption key configured, Neens refuses to store the plaintext — the save succeeds but returns a warning that the credential was discarded, so nothing sensitive is ever written unprotected.
• Instead of pasting a key, you can supply a secret reference (secretRef, e.g. secret://my-key), which Neens resolves at use time from the environment variable NEENS_SECRET_<NAME> on the server — useful when secrets are injected by your infrastructure rather than stored in Neens.

How runs pick a connection

When a judge run, enrichment, insight summary, or clustering pass needs an LLM, Neens resolves the connection in this order, always restricted to connections visible to the run’s agent:

1. The explicit connection chosen in the feature’s configuration (e.g. the judge editor).
2. The agent-visible connection marked default.
3. The first agent-visible connection.

A run can never pick up a connection scoped exclusively to another org or agent — the same visibility rule the Settings page applies on read also governs the workers, so a credential never leaks across an internal boundary (and company-level physical isolation keeps it from ever crossing tenants).

Judges can additionally select multiple connections as a pool: an eval run distributes its scoring calls across the pool round-robin, multiplying throughput across several hosts or API keys. See Judges.

What consumes the connection

Without a connection

An agent with no visible connection degrades gracefully, never errors the platform:

• Clustering still runs — clusters simply stay unlabeled until a connection appears.
• Judge runs, enrichments, and remediation generation report a clear configuration error for that run instead of failing silently.
• Anomaly/regression detectors, retention, and ingest make no LLM calls at all, so observability keeps working end to end.
• The Insights fleet briefing reports unavailable and serves its computed, real-number briefing instead — permanently, with nothing queued and nothing retried. That is a supported end state, not an error.

Reference

Connection changes are recorded to the audit log (llm_connection.create, llm_connection.update, llm_connection.delete).

Troubleshooting

Related

• Judges — the heaviest consumer of connections, including pools.
• Pre-prod evaluations — golden-dataset replays and agent_http agent endpoints.
• Audit log — who changed which connection, and when.