Analysis: The Record of Every Conversation and Skill
What the Analysis section of the admin console shows — the deterministic record of every gateway conversation and every skill invocation — and how to read it.
On this page
TL;DR: Analysis shows what the gateway and the harness hooks recorded — who, which client and model, turns, tool calls, errors and denials, tokens and cache share, cost, latency, skills, installs — for every conversation and every skill. Every figure is a count, a sum or a percentile over what was recorded. Nothing on these pages is an opinion about the work. It starts at /admin/analysis/conversations.
What the section is for
Three questions, in the order you normally ask them:
- What is flowing through the gateway — how many conversations, by whom, on which models and clients, at what cost, with what errors?
- Which skills are actually used, by how many of the people entitled to them, in how many conversations, at what cost, and how do those conversations go?
- What did a new version of a marketplace do, compared with the one before it, and which devices received it?
The pages
Every Analysis page lives at /admin/analysis/<noun>, the same noun and identity the Platform section uses at /admin/<noun> — Platform says what is configured, Analysis says what happened.
| Page | Route | What it answers |
|---|---|---|
| Conversations | /admin/analysis/conversations |
Every gateway conversation in the window: KPI tiles with sparklines, three charts on one time axis, a breakdown by model, client, group, project, person or skill, and the rows themselves — every figure toned, with a tokens-per-turn sparkline per row. |
| Conversation | /admin/analysis/conversations/{context_id} |
One conversation on every plane: per-turn charts (tokens, cost, latency), the turn ledger with model, status, finish reason and tokens by kind, every tool call from the ledger (intent → execution → artifact), every governance decision, every safety finding, and the skills invoked with the marketplace version served at the time. |
| Skills | /admin/analysis/skills, /admin/analysis/skills/{plugin:skill} |
Every skill invoked in the window, grouped by marketplace: invocations with a fourteen-day sparkline, people over entitled, installs, conversations, tokens, cost per invocation, tools, errors and denials, p95, and models and clients seen. Above the table, marketplace adoption: entitled → installed → active, by host. A skill's own page splits its invocations by model, client, group, project, person or version, and lists the conversations behind them. |
| Reports | /admin/analysis/reports, /admin/analysis/reports/{id} |
On-demand AI reports over the record: a headline, an ok / watch / degraded assessment, themes with evidence links and recommendations, written the moment they are asked for from a SQL digest — never a transcript. The latest global report's headline sits in a banner under the Conversations and Skills headers; Report on this view writes one over the page's current filters. |
| Versions | /admin/analysis/versions, /admin/analysis/versions/{marketplace_id} |
Every marketplace by content hash: its history, what changed between versions, how each performed, and what devices received. |
| Tools & artifacts | /admin/tools, /admin/artifacts |
Under AI activity, not Analysis, but counted the same way: every tool call the ledger recorded (kind, server, input summary, state, duration, governance decision, person, session and conversation), and the subset that produced an artifact — something a person can view or retrieve afterwards. |
Every list page shares the same furniture: KPI tiles with an icon and a trend slot, charts padded to the whole window, a filter ribbon whose pills list only the values the filtered set contains (with counts and removable chips), a breakdown whose rows link into the list and download as CSV, row checkboxes that enable a bulk bar (Export selected), and a ? beside the title that opens the page's glossary.
Where the numbers come from
A conversation is a gateway context. The conversation_rollup job keeps one row per context — conversation_facts, with a row per skill it invoked in conversation_skill_facts — re-derived within two minutes of any change on any plane:
- Gateway (
ai_requests): turns and side calls, provider and model per request, input, output, cache-read, cache-write and reasoning tokens, cost, latency percentiles, status and finish reason, client kind, attestation and wire protocol, group and project stamped at insert. - Tool ledger: the model's tool intents joined to their executions and results, so a row can say "asked for 5, executed 4, 1 failed, 2 artifacts". A tool call and an artifact are different things even when they are counted alike:
artifact_kind(schema46_tool_artifacts.sql) marks a call as an artifact only when a person can view or retrieve what it produced — a file the assistant edited, wrote or read (file), an MCP Apps UI resource (ui), a typed card such as a table, chart or report (card), or a retained structured body with a preview (body). A shell command, a search, a listing, a Skill or Task call, or an untyped result is a plain tool call. Thetool_activityview applies the rule to every ledger row, and the Tools page, the Artifacts page, the conversation record and the Skills figures all read it. - Governance: every allow, warn and deny the chain took on the conversation's tool calls; the gateway safety scanners' findings and blocks.
- Hooks: prompts, events and skill invocations from the harness, attached through the Claude Code session id the gateway records on each request.
The Skills page counts invocations from the hooks and reads everything else from the same per-conversation rows, so a figure on a skill is the same figure on the conversations that produced it. People / entitled is distinct invokers over the people the access-control rules reach; installs are distinct consumers holding a verified installation receipt for the skill, on any host; install rate is installed over entitled and activation is active over installed.
Reports
A report is a one-off, on-demand reading of the record by the configured AI model. Requesting one (from /admin/analysis/reports, or Report on this view / Generate in the banner on Conversations and Skills) computes a digest in SQL — totals, the top skills, models and people, cost outliers, denial clusters, the client mix — stores it as the request's inputs, and starts one structured call to the model on a background task. Nothing from a transcript is included and nothing is scheduled. The report page shows a loading widget and refreshes itself when the result lands, usually within 10–40 seconds; a failed call is recorded with its reason and Retry starts a fresh one. Every version stays in the history with the request id, tokens and cost of its call, and a report can be regenerated over the same scope and window at any time.
A report is a reading aid over the figures. The figures stay the record.
Versions
A marketplace version is the sha256 of everything it delivers — its own config, every plugin it includes and every skill those plugins ship. It is recorded at boot and on every inventory sync, and a conversation is tied to the version being served when it first invoked one of the marketplace's skills, whatever is deployed later. Versions has three views per marketplace — History, Compare and Distribution — described in Versions: Marketplace Hashes, History and Compare.
Access
Conversations and Versions are open to marketplace participants for the people in their scope; Skills and Reports need console access. The manual review form and the withdrawal decisions on a marketplace's Distribution view require manage rights; without them the pages render read-only. See Users, Roles and Access.
Troubleshooting
A conversation shows no skills, tools or governance
Symptom: The row has turns, tokens and cost, but the skills, tools and governance columns are empty. Cause: The gateway recorded no Claude Code session id on its requests, so nothing from the hook plane can attach. Plain API clients and some third-party harnesses have no hooks. Solution: Nothing to fix for those clients; the gateway columns are complete. For Claude Code, check the plugin carrying the governance hooks is installed.
A skill has invocations but no cost
Symptom: The skill row counts invocations and people, but conversations, tokens and cost are zero. Cause: The sessions that invoked it produced no gateway conversation with the same session id — the inference went elsewhere, or the client did not send its session id. Solution: See Create a Conversation and See It Land.