2.3 Memory & context

The naive loop from 1.1 has a habit that seems harmless and turns out to be fatal: every step, it appends the new turn to messages and sends the whole thing again. That’s failure mode #3 — context grows forever.

Remember from 0.1 that the model re-reads its entire context on every single call. So an ever-growing messages array means each step is slower, more expensive, and — past a point — dumber, as the important bits get lost in the noise.

Watch context outgrow the window

Below, each turn of work adds ~220 tokens. Keep pressing Add a turn with hydration off and watch the context you send the model climb until it blows past the window. Then flip Hydrate context on and run the same task again.

Hydrate context (summarize old turns, keep the last 3)strategy: append everything

History (everything)

320

tokens · always grows · lives in the durable log

Context sent to the model

320

tokens · what you pay for & what the model reads

Context window budget320 / 2,600

1 turns of work done

Keep adding turns. Because every turn is appended, the context grows in lockstep with history — and around 12 turns it blows past the window. Now flip on Hydrate context and watch the same task stay cheap.

Off, the context tracks history one-for-one and soon overflows. On, the same long task keeps a flat, cheap context — because old turns get folded into a short summary and only the most recent few are sent in full.

Three different things (that beginners conflate)

The fix starts by refusing to treat “the conversation” as one blob. It’s three:

Concept	What it is	Where it lives
History	Everything that happened, in order	The durable event log (from 2.1)
State	A running summary of the older history	A small `summary` string
Context	What the model actually sees this turn	Rebuilt on demand, every step

History is the complete record — you never throw it away. But you don’t feed it to the model. Instead, each step you hydrate a fresh context: the system prompt, the pinned task, the summary of old work, and just the last few turns verbatim.

What that looks like in code

function buildContext(state) {
  return [
    { role: 'system', content: SYSTEM_PROMPT },
    { role: 'user', content: state.task },          // the pinned goal
    { role: 'system', content: `Summary so far: ${state.summary}` },
    ...state.recentTurns,                            // only the last few, verbatim
  ];
}

// When recent turns get too big, fold the oldest into the summary and drop them.
async function compact(state) {
  if (tokens(state.recentTurns) < BUDGET) return state;
  const [old, keep] = splitOldest(state.recentTurns);
  state.summary = await summarize(state.summary, old); // one cheap model call
  state.recentTurns = keep;
  return state;
}

buildContext is the whole idea in one function: hydrate a small, relevant context each step. compact keeps it from growing by summarizing the past. The full history is still safe in the event log if anything ever needs it.

In production

This discipline has a fashionable name now — context engineering — and a toolbox:

Vector stores / retrieval to pull only the relevant past facts into context.
Summarization / memory modules in agent frameworks that compact history for you.

You built the core move by hand: separate history from context, and hydrate on demand.