AI Cost Tracker — Pricing Sources

The cost-tracker uses a three-layer hybrid pricing pipeline. Understanding the priority order is essential for accurate billing attribution.

Priority order

override  (highest — set manually by admin)
  yaml    (fallback — versioned in repo for non-LLM providers)
  litellm (lowest — auto-synced daily from open source)

compute_cost(provider, model, units, ts) returns the cost using the highest-priority active price for the given model at timestamp ts.

Layer 1 — LiteLLM auto-sync (LLM providers)

Source: https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json

Coverage: Gemini, OpenAI, Anthropic, Cohere, Mistral, Together, and 100+ others.

Sync schedule: Celery beat task tasks/pricing_sync.py runs daily at 02:00 UTC.

What happens on sync:

Fetches the JSON (pinned to a configurable commit hash in LITELLM_COMMIT_HASH env var)
Maps input_cost_per_token / output_cost_per_token to our schema
For each model:
- If no existing price → creates new model_prices record with source="litellm"
- If price unchanged → updates litellm_commit_hash, no other change
- If price changed → closes old record (effective_to=today), creates new record, sends admin notification email

Note: Auto-price changes never silently modify source="override" records. If you have an override active, the sync logs a divergence warning but leaves your override intact.

To pin a specific version (for reproducibility):

LITELLM_COMMIT_HASH=abc123def456   # pin to a known-good commit

See wiki/concepts/litellm-pricing-source for deeper explanation.

Layer 2 — YAML (non-LLM providers)

File: backend/app/pricing/models.yaml — versioned in the cost-tracker repo.

Contains providers that LiteLLM does not cover:

# ElevenLabs
- provider: elevenlabs
  model: eleven_multilingual_v2
  billing_unit: char
  price_per_1k_usd: 0.30
  effective_from: "2025-01-01"

- provider: elevenlabs
  model: eleven_flash_v2_5
  billing_unit: char
  price_per_1k_usd: 0.11
  effective_from: "2025-01-01"

# Google Cloud TTS
- provider: google_tts
  model: standard
  billing_unit: char
  price_per_1m_usd: 4.00
  effective_from: "2024-01-01"

- provider: google_tts
  model: wavenet
  billing_unit: char
  price_per_1m_usd: 16.00
  effective_from: "2024-01-01"

When to update YAML:

ElevenLabs raises/lowers per-char pricing
Google Cloud TTS changes tier pricing
Adding a brand-new non-LLM provider

How to update:

Add a new entry with the new price and effective_from: "YYYY-MM-DD"
Leave the old entry — it is used for historical cost attribution
Deploy the new YAML → loader upserts on startup

Do NOT delete old entries — they are needed for retroactive reports.

Layer 3 — Admin override (UI)

Use when you have:

A negotiated enterprise contract price (different from public pricing)
A volume discount or committed-use agreement
A temporary promotional rate
A price correction before the next LiteLLM sync

How to create an override:

Admin UI → Pricing → find the model → Override price
Set: price_per_unit_usd, effective_from (defaults to today), optional override_reason
Save → old price gets effective_to=effective_from, override is now active

Override records are never auto-modified by LiteLLM sync.

Historical pricing and retroactive reports

Every usage event is stored with price_id — a reference to the exact model_prices record active at the time of the call:

Retroactive reports are always accurate — changing a price today does not affect yesterday's costs
Old model_prices records with effective_to set are never deleted
Re-evaluating historical costs with new pricing = manual export + spreadsheet (not a built-in feature)

Monthly reconciliation

Recommended monthly check:

Download invoice from Google Cloud Console / ElevenLabs dashboard
Compare with cost-tracker "Actual vs Billed" report (Admin UI → Analytics → Reconciliation)
If >5% discrepancy: check for pricing_missing=true events and add missing model prices

4.3 KiB Raw Blame History