Architecture blueprint
Define a layered context stack: immutable system rules → task instructions → retrieved knowledge → user profile → conversation state → tool results → temporary working memory.
Pathway 3 · Lesson 0460 min
Context Engineering at Scale
Designing information environments for production AI
Learning Objective
By the end of this lesson, you will be able to: Design a production context pipeline with system prompts, dynamic retrieval, and context window management for a real organisational use case.
200K+
token context windows — but bigger isn't always better
Engage
"You can now give AI a 200,000-token context window — roughly 150,000 words. But research shows that more context often produces worse results. Why? And what does good context engineering actually look like?"
Explore
A practical overview of selecting, compressing, and structuring context for reliable production AI systems.
Explain
The Lost in the Middle Problem
Research from Stanford (2023) found that LLMs perform significantly worse on information placed in the middle of long contexts — they pay more attention to the beginning and end. This is called the 'lost in the middle' problem.
Implication for context engineering: Don't just dump all available context into the prompt. Curate carefully. Place the most important information at the beginning or end of the context. Use retrieval to find the most relevant chunks rather than including everything.
Explain
System Prompt Design Principles
The system prompt is the most powerful lever in context engineering — it persists across all interactions and defines the AI's fundamental behaviour.
Effective system prompt structure:
1. Persona: Who is the AI? What is its role and expertise?
2. Scope: What topics is it authorised to address?
3. Constraints: What must it never do or say?
4. Format: How should it structure its responses?
5. Escalation: When should it defer to a human?
Anti-patterns: Vague personas ('be helpful'), no constraints, no escalation paths, contradictory instructions.
Elaborate · Worked Example
Production System Prompt: Legal Research Assistant
Context
A law firm wants an AI assistant for junior lawyers doing initial case research.
The Situation
Persona: 'You are a legal research assistant for [Firm Name]. You have expertise in UK commercial law.' Scope: 'You answer questions about case law, legislation, and legal principles. You do not provide legal advice to clients.' Constraints: 'Never state a definitive legal conclusion. Always note that findings require review by a qualified solicitor. Never discuss matters outside UK commercial law.' Format: 'Structure responses as: Summary (2-3 sentences), Relevant Cases (bullet list with citations), Key Principles (numbered list), Caveats (what further research is needed).' Escalation: 'If asked for advice on a specific client matter, respond: This requires review by a qualified solicitor. Please consult your supervising partner.'
The Outcome
This system prompt creates a useful, bounded, safe research tool that augments junior lawyers without creating liability.
Try It · Reflect & Apply
Write a system prompt: Design a system prompt for an AI assistant in your organisation. Include all five elements: Persona, Scope, Constraints, Format, and Escalation. Test it mentally — what would it do well? Where might it fail?
Your notes are saved in your browser for this session.
Evaluate · Knowledge Check
1 / 3What is the 'lost in the middle' problem in LLM context windows?
Builder Engineering Studio
Context Engineering Studio
Engineering trade-off
More context is not always better. Curated, current, structured context outperforms a large unfiltered prompt, especially when key information gets lost in the middle of a long window.
Quality gate
Version your system prompts, evaluate any prompt change against a fixed test suite, and reject changes that improve one scenario while harming safety, accuracy, or tone elsewhere.
Evidence to measure
Track context length, retrieval hit rate, instruction-following score, latency, cost, correction rate, and the rate at which the system correctly says it does not know.
Build task
Write a production context specification for one use case: source precedence, information allowed in context, exclusions, compression policy, and escalation triggers.
External video references
Keep Learning on YouTube
Open in a new tab
Context Engineering for AgentsEmbedded lesson videoA practical overview of selecting, compressing, and structuring context for reliable production AI systems.Watch · Watch on YouTube Context Engineering for AgentsLangChainA practical overview of context selection, compression, isolation, and memory strategies for production AI systems.Watch · Technical talk
Human-centred work design
Human Oversight Check
Use this original practice tool to test whether an AI-enabled workflow supports real human judgment, rather than merely placing a person at the end of an automated process.
Self-check
0/5
Start by making the work and decision rights explicit
Calibrate autonomy
Match AI freedom to consequence. Low-consequence drafting may run quickly; high-consequence decisions need explicit human approval.
Keep context visible
Make the objective, inputs, constraints, source material, and current system state easy to inspect before anyone approves an output.
Design active review
Review should require informed judgment, a plausible override, and the authority to act—not a symbolic sign-off after the fact.
Route exceptions clearly
Specify what uncertainty, conflict, missing evidence, or risk looks like—and who receives the escalation with enough context to decide.
Maintain human capability
Protect deliberate practice in the work that people may one day need to verify, challenge, or perform without AI assistance.
Apply it to one workflow
Ready to move on?
Mark this lesson complete and continue to "AI Agent Architecture".