LLM Cost Tracker plugin thumbnail

LLM Cost Tracker

Privacy-conscious LLM usage and cost tracking with a local SQLite dashboard. Tracks chat and utility model calls, separates provider-reported, estimated API, and API-equivalent costs.

Author Jehu 1 stars Version 0.4.0 Updated
LlmToolsDevelopmentAutomation
GitHub Repository Discussion How to install

README

LLM Cost Tracker

Know exactly what your AI costs. A privacy-first plugin that records token usage and estimates costs across all your LLM calls — chat, subordinates, utility models, and external tools like OpenRouter.

What it does

Every time Agent Zero talks to an LLM or AI API, tokens are consumed and money is spent. This plugin tracks both — automatically, locally, and without sending data anywhere.

Feature What you get
📊 Usage dashboard Token counts and costs per model, provider, agent role, and time period
💰 Three cost perspectives Provider-reported, estimated from your price catalog, and API-equivalent comparison
🔒 Privacy-first No prompts or completions are stored — only token counts and cost metadata
🏷️ Price management Add your own pricing rules; the plugin never invents or fetches prices
🔍 Provider discovery See which providers and models you use, and which ones lack price rules
📈 Export Download usage data as JSON or CSV for external analysis
🔗 Plugin integrations Automatically captures costs from compatible plugins like OpenRouter Gateway

Quick start

  1. Enable the plugin — go to Settings → Developer → LLM Cost Tracker and toggle it on.
  2. Open the dashboard — click LLM Cost Tracker in the sidebar quick actions.
  3. Use Agent Zero normally — every chat, subordinate, utility, and tool call is tracked automatically.

That's it. The dashboard populates as you use Agent Zero.

Understanding the three cost types

The plugin deliberately keeps three monetary concepts separate so you never confuse a real charge with an estimate:

Cost type What it means When it's available
Provider-reported The actual cost your provider reports in the API response Only when the provider includes structured cost data (e.g. OpenRouter)
Estimated Calculated from a price rule you configured When you add a matching entry in Settings with a valid price source
API-equivalent What it would cost on OpenAI's API for comparison When you configure an equivalent OpenAI price rule

If none of these apply, the cost stays Unknown — the plugin never guesses.

Dashboard overview

The dashboard shows:

  • Summary cards — total calls, tokens, and costs across all recorded events
  • Provider breakdown — usage grouped by provider class and model
  • Cost status badges — quickly see which models have price rules and which don't
  • Recent events — the latest tracked calls with timestamps and details
  • Export — JSON or CSV download for external reporting (up to 5,000 rows)

Adding pricing rules

The plugin ships with no prices — you add them from authoritative sources.

Via Settings UI

  1. Go to Settings → Developer → LLM Cost Tracker.
  2. Find the Pricing section.
  3. Either use Add missing-price template (auto-generates placeholders for models you've used) or paste JSON into the Advanced pricing editor.
  4. Fill in rates from your provider's official pricing page.
  5. Click Apply JSON, then Save.

JSON format

[
  {
    "id": "openrouter-flux-schnell",
    "provider_class": "openrouter",
    "model": "fal-ai/flux/schnell",
    "input_usd_per_million": 0,
    "output_usd_per_million": 0.04,
    "source_url": "https://openrouter.ai/models/fal-ai/flux-schnell",
    "valid_from": "2025-01-01"
  }
]

Required fields per price rule

Field Description
id Unique identifier across all rules
provider_class Provider type: openai_api, openrouter, zai_api, zai_coding, or other
model Exact model name or * for all models in that class
input_usd_per_million Cost per 1M input tokens in USD
output_usd_per_million Cost per 1M output tokens in USD
source_url Absolute URL to the pricing page you used
valid_from ISO date when these prices took effect

Optional: valid_until, cached_input_usd_per_million.

What gets tracked?

Call type call_role label How usage is captured
Main agent chat turns chat From LLMResult.usage via framework hook
Subordinate agent turns subordinate Same hook, different agent number
Background model calls background Same hook, background flag
Utility model calls utility / utility_background Token estimate from text (framework limitation)
OpenRouter image/TTS/etc. openrouter_image, openrouter_tts, … Via plugin integration (when both plugins are active)

OpenRouter integration

When the OpenRouter Gateway plugin is also installed and active, costs from these endpoints are automatically captured:

Endpoint Data captured
Image generation (/images) Body usage + response headers
Text-to-speech (/audio/speech) Response headers (raw byte responses)
Transcription (/audio/transcriptions) Body usage data
Embeddings (/embeddings) Body usage data
Video generation (/videos) Body + headers at submit and completion

No configuration needed — both plugins discover each other automatically. If the Cost Tracker is disabled, the OpenRouter plugin continues working normally.

Privacy and data minimization

Concern How it's handled
Prompts and completions Never stored — only token counts and metadata
API keys and credentials Filtered at every depth by recursive sanitization
Raw usage data Disabled by default; enable in Settings if needed
Data location Local SQLite database inside the plugin directory — nothing leaves your machine
Data retention Configurable via retention_days in Settings

When raw usage storage is enabled, sensitive keys (api_key, authorization, token, password, cookie, etc.) are recursively stripped, strings are truncated, and payloads exceeding the byte limit are replaced with a truncation marker.

Settings reference

Setting Default Purpose
enabled true Master on/off switch
store_raw_usage false Store detailed usage JSON (sanitized)
retention_days (unlimited) Automatically delete events older than N days
custom_prices [] Your pricing rules array
raw_usage_fields (default set) Which usage fields to keep when raw storage is on
raw_usage_max_bytes 4096 Maximum serialized size for raw usage JSON

Troubleshooting

Symptom Fix
Dashboard is empty Use Agent Zero (send a message) — events appear in real time
All costs show "Unknown" Add pricing rules in Settings — see Adding pricing rules above
OpenRouter costs not appearing Ensure both plugins are enabled; check that the OpenRouter Gateway version supports cost tracking
Plugin doesn't appear Reload the WebUI after enabling

Development checks

cd /a0
/opt/venv-a0/bin/python -m unittest discover -s usr/plugins/llm_cost_tracker/tests -v
/opt/venv-a0/bin/python -m compileall -q usr/plugins/llm_cost_tracker
node --check usr/plugins/llm_cost_tracker/webui/dashboard-store.js

Removal

Delete the plugin directory to remove the plugin and its database. It creates no services, symlinks, or state outside its directory.

License

MIT. See LICENSE.