LiteLLM as Pricing Source

The AI Cost Tracker uses the open-source LiteLLM model prices JSON as the primary source of truth for LLM pricing. This eliminates the need to scrape provider websites.

What is LiteLLM?

LiteLLM is an open-source Python library (30k+ GitHub stars) for calling 100+ LLM providers with a unified interface. It maintains a community-curated model_prices_and_context_window.json covering Gemini, OpenAI, Anthropic, Cohere, Mistral, Together AI, and many others.

Why not scrape provider websites directly?

Problem	Impact
Pricing pages are React SPAs	Need headless browser; brittle
Layout changes without notice	Breaks silently; wrong costs logged
Different billing units per provider	Complex parsing; easy to get wrong
Tier/volume discounts in HTML	Nearly impossible to parse reliably
ToS may prohibit scraping	Legal risk

LiteLLM maintains all of this in a single structured JSON — battle-tested by thousands of production deployments.

The JSON structure

{
  "gemini/gemini-3-pro-preview": {
    "input_cost_per_token": 0.00000125,
    "output_cost_per_token": 0.000005,
    "litellm_provider": "google",
    "mode": "chat",
    "max_tokens": 65536
  }
}

What LiteLLM does NOT cover

ElevenLabs — not an LLM; character-based billing
Google Cloud TTS — not an LLM; character-based billing
Self-hosted models — no external billing

These are defined in pricing/models.yaml in the cost-tracker repo. See wiki/tech-patterns/cost-tracker-pricing-sources.

Keeping prices accurate

Daily Celery beat task (tasks/pricing_sync.py) fetches the latest JSON
If a price changes → admin gets notified; new price record created with effective_from=today
Old price records kept forever for historical reporting
To freeze at a known-good version: set LITELLM_COMMIT_HASH env var

The alternative considered

Direct website scraping was evaluated and rejected due to the problems listed above. LiteLLM is the standard community solution for this exact use case.

2.3 KiB Raw Blame History