AI
Context Windows
A practical guide to understanding what context windows are and why they matter for AI product design.
What a context window is, how it affects AI behaviour across a conversation, and what product and design teams need to account for when building AI features.
What it is
A glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term window is the amount of text an glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term can glossaryProcessA process is a defined sequence of steps used to achieve a specific outcome.Open glossary term and consider at one time. It defines the model's effective working memory for a given conversation or task.
Everything the glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term can see (the glossarySystemA system is a collection of interconnected components that work together to achieve a specific function or outcome.Open glossary term glossaryPromptA prompt is the input or instruction given to an AI system to guide its output or response.Open glossary term, the conversation history, any documents provided, and the current message) must fit within the context window. Anything outside it is invisible to the model.
glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term windows are measured in tokens, which roughly correspond to words or parts of words. A context window of 100,000 tokens can hold roughly 75,000 words: a substantial amount, but still finite.
When a conversation exceeds the glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term window, earlier content is dropped. The glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term can no longer reference it, which can cause it to repeat itself, lose track of instructions, or appear to forget what was discussed.
Understanding glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term windows helps you design AI glossaryFeatureA feature is a specific piece of functionality within a product that delivers value to users. It represents something users can do or experience as part of the overall product.Open glossary term that behave reliably across the full range of intended glossaryInteractionInteraction refers to any action a user takes within a product and how the system responds. It includes clicks, taps, gestures, and inputs that drive the user experience.Open glossary term.
When to use it
Understand when glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term window limits become a real constraint. They matter most when:
They matter less when:
Key takeaway
Context window limits are a design constraint, not just a technical one. Plan for them from the start rather than discovering them in testing.
How it works
The basic mechanism
The glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term window is a fixed-size buffer that holds all the text the glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term can currently see. As a conversation grows, new content is added and, once the limit is reached, the oldest content is removed to make space.
This means the glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term cannot look back beyond what fits in the window. It is not that the model has forgotten. The information is simply no longer present in what it can see.
Different glossaryModelA model is a system or representation used to process data and generate outputs, often trained to perform specific tasks.Open glossary term have different glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term window sizes. Larger windows allow for longer conversations and more context, but also increase the computational cost of each glossaryResponseA response is the data or result returned by a server after receiving a request.Open glossary term.
What this means for designers and product teams
Long conversations, lengthy documents, and large guideSystem PromptsWhat system prompts do, how they define an AI's role and constraints, and what product and design teams need to know when working with them.Open guide all consume glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term space. Understanding this helps you make smarter decisions about what to include and when.
For glossaryFeatureA feature is a specific piece of functionality within a product that delivers value to users. It represents something users can do or experience as part of the overall product.Open glossary term that require memory across very long glossaryInteractionInteraction refers to any action a user takes within a product and how the system responds. It includes clicks, taps, gestures, and inputs that drive the user experience.Open glossary term, additional glossaryArchitectureArchitecture refers to the structure and organisation of a system, including how components interact and are designed.Open glossary term (such as summarisation of earlier content or persistent storage) may be needed to supplement the context window.
What to look for
Focus on:
Where it goes wrong
Most issues come from: Designing a conversational AI glossaryFeatureA feature is a specific piece of functionality within a product that delivers value to users. It represents something users can do or experience as part of the overall product.Open glossary term without accounting for glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term limits will produce glossaryReliabilityReliability is the ability of a system to consistently perform as expected without failure.Open glossary term issues at scale.
What you get from it
Understanding glossaryContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term windows gives you:
Key takeaway
Context windows define what the AI can see. Design your features around that constraint rather than against it.
Get in touch
When an AI feature forgets what it should be holding on to, the context window is the first place to look. We can help you decide what belongs in it.
No guesswork. No assumptions. Just a context window doing useful work.
FAQ
Common questions
A few practical answers to the questions that usually come up around this method.
What is a context window in AI?
It is the amount of text an AI model can process at one time. Everything the model can see (the conversation history, any documents provided, and the current message) must fit within this window. Content that exceeds it is no longer visible to the model.
Why does AI seem to forget things in long conversations?
Because the earlier parts of the conversation have moved outside the context window. The model has not forgotten: it simply cannot see content that is no longer within its working memory.
How big are context windows?
It varies by model and is increasing over time. Many current models support context windows of 100,000 tokens or more, which is roughly equivalent to a short novel. But even large windows have limits, and the more content you put in, the less reliably the model attends to all of it.
Can AI remember things between separate conversations?
Not through the context window: it resets at the start of each new conversation. Persistent memory across sessions requires additional architecture, such as storing summaries or relevant details and injecting them into new conversations.
Should I always use the largest available context window?
Larger context windows give more flexibility, but they are more expensive to run. For short, focused interactions, a large context window offers no benefit. Match the context window size to the actual requirements of the feature you are building.
Quick take
If an AI seems to forget earlier parts of a conversation or loses track of what was said, the context window is usually why.
Related Services
