Dashboards
When the built-in Overview and Agent home don’t show exactly what your team watches, build your own dashboard: pick measures from the metrics catalogue, slice them by dimension, and share the result with exactly the audience you intend. Neens also ships four ready-made persona dashboards — executive, product-quality, finance, and org-scorecard views — that work on every plan.
At a glance
| Where | The Dashboards page (your boards as cards, with a Favorites strip on top) |
| Key API routes | GET/POST /dashboards, POST /dashboards/{id}/widgets, GET /dashboards/{id}/widgets/{wid}/data, POST /dashboards/{id}/clone, GET /dashboards/by-slug/{slug} |
| Data source | Every widget resolves live through the metrics catalogue — no widget can query outside it |
| Scope | A dashboard is pinned to an agent set, an org, or the whole workspace; viewers only ever see data from agents they can access |
| Plan | Creating and cloning dashboards requires the Silver plan or above; viewing any dashboard (including the platform persona dashboards) works on every plan |
Create a dashboard
New dashboard
On the Dashboards page, click New dashboard, give it a name, and pick a scope & sharing setting (see Scope and visibility). If you create it while an agent is selected, it defaults to that agent’s scope and agent visibility.
Add widgets
Click Add widget. Either pick a ready-made preset from the gallery (one click drops in a fully configured chart, organized under Volume · Quality · Latency · Errors · Cost · Topics · Business tabs — the Business ones are measured per case and need a KPI definition first), or use the Custom tab to choose:
- a measure — what to chart (traces, error rate, latency, spend, scores, …),
- optional dimensions — what to break it down by (time, agent, model, your own metadata fields, …),
- a visualization — one of
kpi,timeseries,bar,table,heatmap,donut.
The builder only offers valid measure × dimension combinations — the full menu is documented in the metrics catalogue. Invalid combinations are rejected by the API (422), so a broken widget can never be saved.
Arrange
Widgets flow in order across a grid: drag a widget’s edges to resize, drag its grip handle to reorder, or duplicate/remove it from its menu.
Per-widget accent color
In the Custom tab, pick a Color for the widget: the theme Default, a curated swatch, or your own hex value (#RGB or #RRGGBB — anything else is rejected). Single-series charts use the accent directly; a donut derives a harmonized multi-color palette from it. Tables and heatmaps ignore color.
Per-widget scope
By default a widget inherits the dashboard’s scope. You can override an individual widget to its own scope — for example, one tile pinned to a single agent on an org-wide dashboard. A badge on the widget shows when it has its own scope. Widget scope overrides are access-checked the same way dashboard scopes are.
Time ranges — the precedence
Every dashboard has a shared time picker in its header with the standard presets (Today · Last 24h · 7 days · 30 days · All time · Custom). Widgets resolve their window in this order, most specific first:
- The widget’s own pinned range — an authoring choice that fixes one tile to a window. It beats even the header picker, so a deliberately pinned panel stays put.
- The header picker selection — drives every non-pinned tile, so the whole board shares one window.
- The dashboard’s saved default range — the board’s initial window (one of
today,24h,7d,30d,all), used when no interactive selection is supplied. Editors set it from the dashboard header. - The measure default — all-time (no bound).
When a dashboard has no saved default, the picker starts at the last 30 days.
Scope and visibility
A dashboard’s scope pins what data it shows; its visibility controls who can see it. They’re related but separate settings:
| Scope | Data shown |
|---|---|
| Agent | One or more pinned agents |
| Org | All agents in one org (resolved live — new agents appear automatically) |
| Tenant | The whole workspace |
| Visibility | Who sees the dashboard |
|---|---|
| Private | Only the owner |
| Agent | Anyone with access to at least one of the pinned agents |
| Org | Anyone with access to that org |
| Company | Everyone in the workspace |
Sharing is permission-gated: any member can create dashboards and share to agent or org visibility, but sharing company-wide requires an admin role. Deleting someone else’s dashboard also requires admin; owners can always delete their own.
Sharing a dashboard never shares data. Widget queries are always constrained to the viewer’s accessible agents at render time. A company-visible dashboard scoped to agents a viewer can’t access simply renders no data for them — there is no way to leak another team’s numbers through a shared board.
Favorites, cloning, deleting
- Favorite (star) a dashboard to float it into the Favorites strip at the top of the Dashboards page.
- Clone a dashboard — from the card’s hover action or the header Clone button — to get a private copy you own, with every widget’s configuration (including colors and scope overrides) deep-copied. A clone never inherits the original’s audience: it always starts private, and any scope pins you can’t access degrade to what you can see. Favorites aren’t carried over.
- Clone a widget in place to duplicate one tile within the same dashboard — it appends at the end at the default size, ready to tweak.
- Delete a dashboard from its header (owner or admin). This can’t be undone.
Project Home
Project Home is the landing page for the project you have selected — the built-in read-out of what your agent has been doing. Unlike a custom dashboard, its layout is fixed: you can’t add or rearrange tiles, but it needs no setup and is the same for every project. Top to bottom:
- KPI tiles — Traces, Sessions, Total Spans, Session p50 / p99 and LLM p50 / p99 latency, and Failure Clusters. Once you’ve shipped fixes, Remediations applied and Remediations pending tiles appear too.
- Top failure modes and Open issues — two side-by-side lists linking into Failure clustering and Issues & failure modes.
- Four trend charts — Trace Volume, Latency Trend, Token usage over time, and Traces by model (described below).
Hover any chart to read exact values at a point in time.
Time range
Project Home has a time picker in its header with the standard presets — Today · Last 24h · 7 days · 30 days · All time · Custom — and opens on the last 30 days. The KPI tiles, both charts, and the open-issues list all follow the selected window. The Failure Clusters tile and Top failure modes list are a snapshot of the latest clustering run rather than a windowed count, so they don’t change with the picker.
Token usage over time
A stacked-area chart of the tokens your agent used across the window, split into two bands:
- Input tokens — the tokens sent to the model (your prompts, context, and tool results).
- Output tokens — the tokens the model generated back.
They stack, so the top of the shaded area is total token usage at each point, and the two bands show how that total splits between prompt and completion. Use it to spot cost and usage spikes: a sudden jump in the total height means your agent burned far more tokens than usual, and which band grew tells you why.
For example, if a prompt-template change accidentally pastes an entire document into every request, the input band roughly doubles while the output band stays flat — you’ll see the stack jump in height on the day the change shipped, with all the growth in the lower band. That points you straight at the prompt rather than at the model’s responses. Token usage tracks closely with spend — see Cost & model pricing to turn tokens into dollars.
Traces by model
A bar chart counting traces by the LLM model each one used — one bar per model, tallest first — so you can see your model mix at a glance. Two buckets need a word of explanation:
Mixed counts a single trace that called two or more different models (for example, a cheap model to route the request and a stronger one to write the final answer). Such a trace is counted once, in Mixed, instead of being added to every model it touched — so each named model’s bar stays a clean count of traces that used only that model.
Unknown counts traces with no model attribution — the trace didn’t record which model was used (common when an integration doesn’t set the model on its spans). It isn’t an error; it just means Neens couldn’t attribute the trace to a specific model.
Platform persona dashboards
Neens ships four platform-managed dashboards, one for each of the leadership-facing persona lenses. They are seeded into every workspace, render on every plan, and are kept up to date by the platform:
| Dashboard | Audience | What it shows |
|---|---|---|
| Executive digest | Executives | Traces, error rate, eval pass rate, and spend — as headline KPIs, trends over time, and per-org breakdowns |
| Product quality board | Product managers | Eval pass rate, average score, score volume — trends, per-judge averages, and the score distribution |
| Finance cost explorer | Finance | Total spend, input/output tokens, average tokens per trace — sliced by model and org, and over time. Spend uses your model price table; widgets whose traffic includes an unpriced model are labelled partial |
| Org scorecard | Org leads | Traces, error rate, and eval pass rate compared across orgs |
All four are tenant-scoped (whole-workspace data), visible company-wide, and default to a 30-day window.
Platform dashboards are immutable to your workspace. You can’t edit, rearrange, or delete them — any attempt returns 403. To customize one, Clone it: the copy is a normal private dashboard you fully own (cloning requires Silver+ like any dashboard creation).
When a user picks (or is assigned) a persona lens, that persona’s dashboards are automatically pinned to their Favorites, and personas whose home page is a dashboard resolve it by its stable slug (GET /dashboards/by-slug/{slug} — e.g. exec-digest, pm-quality, finance-cost-explorer, org-scorecard), so the right board loads no matter which workspace you’re in.
How it works
- Dashboards and widgets are stored durably per workspace; no data is snapshotted. Every tile fetches live at render time (
GET /dashboards/{id}/widgets/{wid}/data), resolving its measure through the same query layer the Overview page uses. - The effective scope for a tile is: its own scope override if set, else the dashboard’s scope — then intersected with the viewer’s accessible agents.
- Widget definitions are validated against the metrics catalogue at save time and re-validated at render, so a stale widget referencing a removed combination fails loudly (
422) rather than showing wrong numbers. - Dashboards record who created them and who added each widget, so a shared board shows attribution.
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| New dashboard is rejected | Your plan doesn’t include custom dashboards | Custom dashboards are Silver+; viewing shared and platform dashboards still works on every plan |
| Can’t set Company visibility | Sharing company-wide is admin-only | Ask an admin to change visibility, or share at org/agent level |
| Editing a platform dashboard returns an error | Platform dashboards are immutable | Clone it and edit the copy |
| A shared dashboard shows no data for a teammate | Its scope pins agents they can’t access | Data follows the viewer’s access — grant agent/org membership, or re-scope the dashboard |
| A tile ignores the header time picker | The widget has its own pinned range | Edit the widget and clear its range so it follows the picker |
Related
- Metrics catalogue — every measure and dimension the widget builder offers.
- Cost & model pricing — the prices behind every spend widget, and how to set your own.
- Model comparison — the Quality tab’s model presets: pass rate by model, by agent × model, and by model attribution.
- Insights — automatically detected anomalies, regressions, and issue spikes.
- Dashboards — the built-in Overview, Agent home, and Insights surfaces.