AI
Context Windows
A practical guide to understanding what context windows are and why they matter for AI product design.
What a context window is, how it affects AI behaviour across a conversation, and what product and design teams need to account for when building AI features.
What it is
A glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term window is the amount of text an glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term can glossaryProcessA process is a defined sequence of steps used to achieve a specific outcome.Open glossary term and consider at one time. It defines the model's effective working memory for a given conversation or task.
Everything the glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term can see — the glossarySystemA system is a collection of interconnected components that work together to achieve a specific function or outcome.Open glossary term glossaryPromptA prompt is the input or instruction given to an AI system to guide its output or response.Open glossary term, the conversation history, any documents provided, and the current message — must fit within the context window. Anything outside it is invisible to the model.
glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term windows are measured in tokens, which roughly correspond to words or parts of words. A context window of 100,000 tokens can hold roughly 75,000 words — a substantial amount, but still finite.
When a conversation exceeds the glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term window, earlier content is dropped. The glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term can no longer reference it, which can cause it to repeat itself, lose track of instructions, or appear to forget what was discussed.
Understanding glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term windows helps you design AI glossaryFeatureA feature is a specific piece of functionality within a product that delivers value to users. It represents something users can do or experience as part of the overall product.Open glossary term that behave reliably across the full range of intended glossaryInteractionInteraction refers to any action a user takes within a product and how the system responds. It includes clicks, taps, gestures, and inputs that drive the user experience.Open glossary term.
When to use it
Understand when glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term window limits become a real constraint. They matter most when:
They matter less when:
Key takeaway
Context window limits are a design constraint, not just a technical one. Plan for them from the start rather than discovering them in testing.
How it works
Understand the basic mechanism. The glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term window is a fixed-size buffer that holds all the text the glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term can currently see. As a conversation grows, new content is added and — once the limit is reached — the oldest content is removed to make space.
This means the glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term cannot look back beyond what fits in the window. It is not that the model has forgotten — the information is simply no longer present in what it can see.
Different glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term have different glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term window sizes. Larger windows allow for longer conversations and more context, but also increase the computational cost of each glossaryResponseA response is the data or result returned by a server after receiving a request.Open glossary term.
What this means for designers and product teams. Long conversations, lengthy documents, and large guideSystem PromptsWhat system prompts do, how they define an AI's role and constraints, and what product and design teams need to know when working with them.Open guide all consume glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term space. Understanding this helps you make smarter decisions about what to include and when.
For glossaryFeatureA feature is a specific piece of functionality within a product that delivers value to users. It represents something users can do or experience as part of the overall product.Open glossary term that require memory across very long glossaryInteractionInteraction refers to any action a user takes within a product and how the system responds. It includes clicks, taps, gestures, and inputs that drive the user experience.Open glossary term, additional glossaryArchitectureArchitecture refers to the structure and organisation of a system, including how components interact and are designed.Open glossary term — such as summarisation of earlier content or persistent storage — may be needed to supplement the context window.
What to look for
Focus on:
Where it goes wrong
Most issues come from: Designing a conversational AI glossaryFeatureA feature is a specific piece of functionality within a product that delivers value to users. It represents something users can do or experience as part of the overall product.Open glossary term without accounting for glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term limits will produce glossaryReliabilityReliability is the ability of a system to consistently perform as expected without failure.Open glossary term issues at scale.
What you get from it
Understanding glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term windows gives you:
Key takeaway
Context windows define what the AI can see. Design your features around that constraint rather than against it.
FAQ
Common questions
A few practical answers to the questions that usually come up around this method.
What is a context window in AI?
It is the amount of text an glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term can glossaryProcessA process is a defined sequence of steps used to achieve a specific outcome.Open glossary term at one time. Everything the model can see — the conversation history, any documents provided, and the current message — must fit within this window. Content that exceeds it is no longer visible to the model.
Why does AI seem to forget things in long conversations?
Because the earlier parts of the conversation have moved outside the glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term window. The glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term has not forgotten — it simply cannot see content that is no longer within its working memory.
How big are context windows?
It varies by glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term and is increasing over time. Many current models support glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term windows of 100,000 tokens or more, which is roughly equivalent to a short novel. But even large windows have limits, and the more content you put in, the less reliably the model attends to all of it.
Can AI remember things between separate conversations?
Not through the glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term window — it resets at the start of each new conversation. Persistent memory across glossarySessionA session is a single period of user interaction with a product, from entry to exit within a defined timeframe.Open glossary term requires additional glossaryArchitectureArchitecture refers to the structure and organisation of a system, including how components interact and are designed.Open glossary term, such as storing summaries or relevant details and injecting them into new conversations.
Should I always use the largest available context window?
Larger glossaryContextThe surrounding conditions that shape behaviour and decisions.Open glossary term windows give more flexibility, but they are more expensive to run. For short, focused glossaryInteractionInteraction refers to any action a user takes within a product and how the system responds. It includes clicks, taps, gestures, and inputs that drive the user experience.Open glossary term, a large context window offers no benefit. Match the context window size to the actual requirements of the glossaryFeatureA feature is a specific piece of functionality within a product that delivers value to users. It represents something users can do or experience as part of the overall product.Open glossary term you are building.
Quick take
If an AI seems to forget earlier parts of a conversation or loses track of what was said, the context window is usually why.
Related Services