Glossary · Definition
Context window
The context window is the maximum amount of text (measured in tokens) a model can consider at once — instructions, history, retrieved documents and its own answer.
By Nythrex EngineeringUpdated 1 min read
In plain words
Everything the model “sees” for a request must fit in its context window. Modern models have large windows, but filling them costs money and time, and models don’t always use very long contexts equally well. Bigger isn’t automatically better.
When it matters to you
- Deciding between RAG and putting documents directly in the prompt
- Long conversations and agent runs
- Estimating cost per request
Common pitfalls
- Stuffing the window “just in case”
- Assuming the model pays equal attention to everything
- Ignoring the cost of long prompts at scale
Keep reading
GlossaryTokensTokens are the chunks of text a language model reads and writes; they are how context limits are measured and how most model APIs are billed.Guide11 ways to cut your LLM billLLM cost optimisation: prompt caching, model routing, shorter context, better retrieval, batching and budgets — checked against evals so quality holds.GlossaryRAG (retrieval-augmented generation)RAG is a technique where an AI system first searches your documents or data for relevant passages and then lets a language model answer using them, with cita…
Want a second opinion on your project?
Tell us what you’re building and where you’re stuck. We’ll reply within one business day with the most practical next step — even if that step isn’t us.