Monthly LLM Cost Estimator

Start with a realistic requests per day number — count actual production traffic, not optimistic projections. Set average input tokens to your typical prompt size (system + RAG context + user message). Average output is whatever your responses look like in production.

If you've enabled prompt caching, set the cache hit rate based on your traffic pattern. Repeated system prompts on similar requests = high hit rate (50–90%). Highly varied per-request prompts = low hit rate (5–20%).

Why is the cheapest model highlighted but not always the right answer?

Cost ranking ignores quality. DeepSeek V3 might be 10× cheaper than GPT-4o for your workload but produce worse JSON-mode outputs. Eval first, optimise cost second.

How accurate is the prompt-caching discount?

For OpenAI, 50% off cached input is the documented number. For Anthropic, "up to 90%" depends on cache breakpoints and TTL. The default 50% is a conservative middle ground; tune for your provider.

What about embeddings, vector DB, hosting?

Not included. Those are flat-ish costs (per GB stored, per query). Add them separately. For most LLM-heavy workloads, inference dominates the bill.

How do I think about cost growth at 10× volume?

It's linear at list price, sub-linear with caching. If you hit serious volume (~$10k/mo+), negotiate enterprise pricing — providers will discount significantly.

• Forgetting that output tokens dominate cost in many workloads.
• Sizing on optimistic "10× growth" projections rather than measured traffic.
• Counting only one model when shipping a router that fans out across several.
• Ignoring the long tail — that one user who asks for 30k-token essays.

• LLM Pricing Comparison

  List-price reference for chat-model APIs across major providers, sortable and reviewed monthly.

• LLM Capability Matrix

  Which frontier models support multimodal, vision, audio, JSON mode, and tool calling.

• Per-call LLM Cost Calculator

  Pick a model, paste your prompt, set expected output length, see the per-call cost.

• Rate Limit / TPM Calculator

  Sanity-check whether your traffic stays within RPM and TPM ceilings for a given model and tier.