Glossary · Definition
Prompt caching
Prompt caching lets a model provider reuse the processing of an identical prompt prefix across requests, reducing cost and latency for repeated content.
By Nythrex EngineeringUpdated 1 min read
In plain words
Many requests start with the same long block: system instructions, tool definitions, a reference document. With caching, providers process that prefix once and bill later reuses at a reduced rate. Structure prompts with stable content first to benefit.
When it matters to you
- Long, repeated system prompts
- Multi-turn conversations
- Agents with many tool definitions
Common pitfalls
- Putting variable content before stable content
- Confusing prompt caching with caching answers
- Assuming the same rules across providers
Keep reading
Guide11 ways to cut your LLM billLLM cost optimisation: prompt caching, model routing, shorter context, better retrieval, batching and budgets — checked against evals so quality holds.GlossaryTokensTokens are the chunks of text a language model reads and writes; they are how context limits are measured and how most model APIs are billed.Free toolLLM API cost calculatorEstimate the monthly cost of an LLM feature from requests, tokens, model price tiers and prompt caching. Free, in your browser, with your own prices.
Want a second opinion on your project?
Tell us what you’re building and where you’re stuck. We’ll reply within one business day with the most practical next step — even if that step isn’t us.