LiteLLM as Pricing Source

The AI Cost Tracker uses the open-source LiteLLM model prices JSON as the primary source of truth for LLM pricing. This eliminates the need to scrape provider websites.

What is LiteLLM?

LiteLLM is an open-source Python library (30k+ GitHub stars) for calling 100+ LLM providers with a unified interface. It maintains a community-curated model_prices_and_context_window.json covering Gemini, OpenAI, Anthropic, Cohere, Mistral, Together AI, and many others.

Why not scrape provider websites directly?

Problem	Impact
Pricing pages are React SPAs	Need headless browser; brittle
Layout changes without notice	Breaks silently; wrong costs logged
Different billing units per provider	Complex parsing; easy to get wrong
Tier/volume discounts in HTML	Nearly impossible to parse reliably
ToS may prohibit scraping	Legal risk

LiteLLM maintains all of this in a single structured JSON — battle-tested by thousands of production deployments.

The JSON structure

{
  "gemini/gemini-3-pro-preview": {
    "input_cost_per_token": 0.00000125,
    "output_cost_per_token": 0.000005,
    "litellm_provider": "google",
    "mode": "chat",
    "max_tokens": 65536
  }
}

What LiteLLM does NOT cover

ElevenLabs — not an LLM; character-based billing
Google Cloud TTS — not an LLM; character-based billing
Self-hosted models — no external billing

These are defined in pricing/models.yaml in the cost-tracker repo. See wiki/tech-patterns/cost-tracker-pricing-sources.

Keeping prices accurate

Daily Celery beat task (tasks/pricing_sync.py) fetches the latest JSON
If a price changes → admin gets notified; new price record created with effective_from=today
Old price records kept forever for historical reporting
To freeze at a known-good version: set LITELLM_COMMIT_HASH env var

Gemini Provider Key Gotcha

LiteLLM stores Gemini model prices under the provider key vertex_ai-language-models, not google. If your integration sends provider: "google" in API calls, the cost-tracker's pricing engine won't find a match and returns cost_usd = null.

Fix: add a provider alias in pricing_engine.py that maps "google" → "vertex_ai-language-models". This keeps client code readable ("google") while matching LiteLLM's internal naming:

PROVIDER_ALIASES = {
    "google": "vertex_ai-language-models",
}

def lookup_price(provider: str, model: str):
    canonical = PROVIDER_ALIASES.get(provider, provider)
    return model_prices.get(f"{canonical}/{model}")

Always verify the exact provider key by searching the LiteLLM JSON for your model name before assuming the provider string.

The alternative considered

Direct website scraping was evaluated and rejected due to the problems listed above. LiteLLM is the standard community solution for this exact use case.

3.1 KiB Raw Blame History