Token Dashboard — Local Telemetry for AI Spend

DRAFT — pending Harsh's verification.

This page documents a local telemetry dashboard Harsh built to track his own AI usage across the coding assistants he actually runs. It is a side tool, not part of Memex or any employer system — a personal observability surface, run entirely on his laptop. The design is interesting precisely because the problem is widespread and almost nobody instruments it: AI spend is real money and real context, but each assistant reports it in a different place, in a different shape, with different fidelity, and most users have no unified view of it at all.

1. The problem

A working engineer in 2026 doesn't use one AI assistant; they use a stack: a terminal coding agent (Codex), a different terminal coding agent (Claude Code), an in-editor assistant (GitHub Copilot), plus chat products for planning and research (Claude chat, ChatGPT). Each one bills (or burns context) independently. The result is a classic observability gap:

• Each source has its own usage surface, in its own format, at its own fidelity. There is no single API that says "how many tokens did I burn yesterday, across everything."
• Exact counters exist for some sources and not others. The terminal agents write local logs that include real token counts; the in-editor assistant only logs request events, not tokens; the chat products expose nothing local at all.
• Spend is invisible by default. With no board, there is no feedback loop. You can't change what you can't see, and most people have never seen their own AI usage summarized by day, by source, and by what the work actually was.

This is the same shape as every metrics problem Harsh has spent his career on (see Build Failure Triage and Release Engineering): make the thing visible first, then behavior follows. The dashboard exists to make AI spend visible.

2. The design: one local board over a scrubbed daily-burn file

The system is two pieces, and the split is deliberate:

1. An importer (npm run import:daily) that reads local logs from each source, normalizes them, and writes a single static file: data/daily-burn.json.
2. A Next.js dashboard that reads that static file and renders it — totals by source, a heat calendar, weekly trend, a driver breakdown, and a "what to do next" nudge.

flowchart LR
  subgraph logs["local logs · exact where counted"]
    codex[Codex telemetry db]
    claude[Claude Code sessions]
    copilot[VS Code Copilot logs]
  end
  importer["import:daily\nscrubbed · estimates where not"]
  board["daily-burn.json\ncounts only · static read"]
  dashboard[Next.js dashboard]

  codex --> importer
  claude --> importer
  copilot --> importer
  importer --> board
  board --> dashboard

The central design decision is a fidelity ladder: use exact local telemetry where the logs expose real token counters, and fall back to labeled estimates derived from request counts where they don't — never guessing a number and labeling it exact. Chat products, which expose nothing local, are written as 0 and intentionally excluded, rather than fabricated. The row carries an evidence note on every line so a reader always knows which is which.

The normalized daily row (as documented in the dashboard README) is:

{
  "date": "YYYY-MM-DD",
  "codex_tokens": 0,
  "claude_code_tokens": 0,
  "claude_code_calls": 0,
  "github_copilot_est": 0,
  "github_copilot_requests": 0,
  "claude_chat_est": 0,
  "chatgpt_est": 0,
  "total": 0,
  "driver": "planning",
  "evidence": "Codex exact local telemetry and Claude Code exact local usage"
}

total is the sum of the token fields. The two _est chat fields are structural placeholders — present so the schema stays stable when those sources are eventually wired, but always zero today.

Source fidelity

Reusable pattern: when sources have different fidelity, build a fidelity ladder — exact where you can, conservatively estimated where you must, explicitly excluded where you can't — and stamp each row with an evidence note. A number labeled "estimate" is useful; a guess labeled "exact" is a lie that breaks trust in the whole board.

3. Per-source importers as separate modules feeding one row

The importer is one script with three per-source modules — importCodex, importClaudeCode, importGithubCopilot — each responsible for a single source. They don't know about each other. They all write into one shared date → row map, lazily creating a row for any date they touch; a row that ends up with a zero total is dropped at the end.

flowchart LR
  codex[importCodex] --> rows["shared rowsByDate map\ndate -> codex_tokens,\nclaude_code_tokens,\ngithub_copilot_est,\ndriverTokens, …"]
  claude[importClaudeCode] --> rows
  copilot[importGithubCopilot] --> rows
  getRow["getRow(date) lazily creates the row\ntotal == 0 dropped at write time"] --> rows

The split keeps each source's parsing logic isolated, so adding a fourth source is "write a new importer that calls getRow(date)," not "refactor a monolith." Each module is also free to be as simple or as defensive as its source needs:

• Codex reads a local telemetry database of completed-response events and pulls the usage block — preferring a reported total, else summing the input, output, cached, and reasoning token parts. A workspace hint in the event metadata classifies dashboard/tool work as shipping and everything else as planning.
• Claude Code reads per-session JSONL logs, first collecting each session's title into a lookup, then summing usage (input + cache-creation + cache-read + output) on every assistant response. The session title is reduced to a driver and then discarded — it is never written out.
• GitHub Copilot reads VS Code's Copilot Chat logs, matching successful request events (which carry a channel/kind hint) plus inline-completion 200s. Because the logs expose request events but no token counters, each matched request is assigned a conservative weight by kind and summed into the _est field; the request count itself goes into github_copilot_requests as the exact count.

Copilot estimation weights

The Copilot weights are design constants documented in the dashboard README, not measured usage. They are the importer's stated conservative per-request token estimates:

A wrinkle the README calls out explicitly: newer VS Code Copilot logs sometimes expose only markdown request records with no success/channel line. In that case the importer counts one fallback request per minute after the last day it saw a real success event, weighting each at the 4,000 "other" figure and keeping it labeled as an estimate. The dashboard never presents these as exact.

4. Pacific-day bucketing

Every timestamp is bucketed to a Pacific calendar day (America/Los_Angeles) before it enters a row. This is not a manual UTC offset — the importer formats each timestamp through the timezone-aware Intl.DateTimeFormat so daylight saving is handled by the platform tz database, not by hand. All three importers route through the same localDate helper, so a Codex epoch-seconds event and a Claude Code ISO timestamp land in the same day bucket consistently.

Why a local day and not UTC? Because the question the dashboard answers is "What did I burn on my Tuesday?" — and a Tuesday defined by a 3am Pacific boundary is the wrong answer for someone whose workday is Pacific. Bucketing to the operator's local calendar day is the choice that matches the question the board is meant to answer.

Reusable pattern: *bucket telemetry to the calendar day of the person *who reads the board, using the platform timezone database — never a hardcoded *UTC offset. Off-by-one days from naive UTC bucketing are the most common bug in any "by day" dashboard.

5. Privacy stance: everything local, counts only

The dashboard's trust posture is the same one this whole site is built around (see the PRD's agent trust rules) — except here it's applied to the operator's own data. The rule is simple: content never leaves the machine; only counts do.

Concretely, the importer reads local files and writes a row that contains:

• per-source token counts and one call/request count,
• a total,
• a driver annotation (the one-word "what kind of work" label), and
• a generic evidence note describing the row's provenance.

It deliberately does not write raw prompts, message content, file paths, client names, request IDs, or session titles. The driver is derived from local metadata and then the metadata is discarded — so the row says driver: shipping without ever recording which session or which file. The README's deploy instruction makes this a gate, not a suggestion: deploy only after confirming daily-burn.json is scrubbed.

The chat-product fields are the strongest expression of this stance. Claude chat and ChatGPT have no local log the importer can read, so rather than guess, the importer writes 0 and marks them intentionally excluded. The board undercounts total spend on purpose before it would invent a number.

6. The DORA-dashboards connection

This dashboard is not a one-off. It is the same instinct that produced Harsh's release-engineering metrics work — and it is worth naming the through-line because it is the actual point.

DORA-style metrics (deployment frequency, lead time, change failure rate, mean time to restore) are valuable not because the numbers themselves are interesting but because putting them on a board changes how the org behaves. The Release Engineering arc — moving from bi-weekly to daily releases at a sub-1% rollback rate across a large multi-repo footprint — only became tractable once delivery became legible: visible on a dashboard, instead of folklore in someone's head.

The token dashboard is the same move, applied to a personal cost surface:

• Make it visible. AI spend is a real and growing line item, but it is scattered across five assistants with no shared view. The board collapses them into one Pacific-day row.
• Name the driver, not just the number. A board that says "you spent X tokens" is only marginally useful. A board that says "your top driver this window was research" tells you what to change. The driver annotation is the behavior-change lever, exactly the way change-failure-rate visibility shifts review practice.
• The board closes the loop. The dashboard computes a single driver-keyed "next action" recommendation — Automate the repeatable shipping loop, Condense research before new exploration, Batch review into checklists, Force planning into decisions. That is the dashboard telling the operator, in DORA fashion, "here is the behavior this data suggests you change next."

DORA made delivery legible and shifted how teams ship. This board makes AI spend legible, by source and by driver, with the same goal: behavior change driven by a visible signal.

7. What it demonstrates

• Instrumenting the thing you can't see. Most engineers have no idea what their own AI usage looks like by day. Building the board is the act of turning an invisible cost into a visible signal — the precondition for managing it.
• A fidelity ladder instead of a uniform lie. The willingness to publish exact counts, labeled estimates, and explicit zeros side by side — each stamped with an evidence note — is the same calibrated-honesty posture the rest of this site is built around (see Build Failure Triage).
• Local-first as a trust principle. Everything runs on the laptop; content never leaves; only scrubbed counts do. The privacy stance is not a feature, it is the design.
• The DORA reflex, on a personal surface. The same "make it visible to change it" instinct that drove the release-metrics work, applied to a new cost class. It is a small, concrete instance of the pattern this site is built to demonstrate: turn an implicit, scattered reality into something legible, and let the visible signal change behavior.

Honest gaps

A few things this page does not claim:

• No real usage numbers appear here, by design — the page describes the architecture and the estimation method, not any actual measured burn. The figures in the estimation-weights table are the importer's documented per-request constants, not usage data.
• The board undercounts total spend. Claude chat and ChatGPT are excluded entirely (written as 0), so total is a lower bound on real spend, not a complete picture.
• Copilot estimates are coarse. They are request counts times fixed weights, not measured tokens, and the fallback minute-counting rule is an explicit approximation for logs that lost their success line.
• The importer is a manual daily run (npm run import:daily followed by npm run build), not a continuous collector. A row appears only after the operator runs it.
• No measured behavior change is recorded. The dashboard's premise — that making spend visible shifts how the operator spends — is the thesis, not yet a measured before/after.

Sources

This page was distilled from the token-dashboard project's own documentation and source — primarily the dashboard's README (which documents the row schema, the source-fidelity table, the Copilot estimation weights, the Pacific-day bucketing, and the driver-classification rules) and the importer script and dashboard app, read for architecture only. No source code, no real usage numbers, and no local file paths from the operator's machine appear here; the design is described, not copied, and every architectural claim above traces to that material.