Memory Management for Long-Horizon Agents
A working memory layer decides what an agent sees at each step, which is a different question from what it stores. Here is what breaks when the history gets long, what the memory layer has to do about it, and what one recent set of measurements says about the shape of a working answer: raw traces that survive their own summaries, notes that carry time and provenance, and a retriever that navigates a directory tree instead of ranking embeddings.
A working memory layer decides what an agent sees at each step, which is a different question from what it stores. This piece walks what breaks when the history gets long, the four properties a memory layer needs to fix it, and one system that implements all four and measured the result: a file-system-style workspace where summaries index the raw trajectories they came from, navigated by a small trained retriever that assembles context on demand.
XO Labs Inc. · August 2026
TL;DR: Memory serves two distinct functions — how experience is organized as it accumulates, and how it is selected at query time — and most systems collapse them into one pipeline optimized against one sparse reward. Split them and each half gets a signal it can learn from: a strong model organizes on the write path, off the step path; a small model, trained with a dense evidence-overlap reward, retrieves on it. Under a 1,950-token budget on ALFWorld that scores 56.7% against 29.1% for the best alternative, and a weak manager handed the strongest retriever available went only 21.0 → 23.3: bad organization caps any retrieval policy.
Sources
Every measurement here comes from a single preprint, marked from the paper where it appears; redrawn illustrations are marked redrawn. Full citation and provenance in section 12.
1. The problem: agents outgrow their context
Large language model agents are stateless: every fact, observation and tool output must be resupplied through the input context on each turn. The context grows with the interaction and the agent carries its whole history forward regardless of relevance, so reasoning quality falls as signal is buried, cost rises on tokens paid for repeatedly, and latency compounds across sequential steps. The standard remedies do not solve this cleanly: compression is lossy and irreversible, and similarity-based retrieval flattens memory into an unordered pool where semantic closeness stands in for the causal and temporal relations agentic reasoning depends on.
Underneath both, one pipeline handles two distinct jobs — organizing experience as it accumulates, and selecting from it at query time — so the storage format ends up dictating what retrieval can do, down to a fixed budget bearing no relation to how much context a query needs, and the result is hard to inspect and harder to improve. What that calls for is a memory layer that keeps raw experience reachable, preserves structural, temporal and causal relationships instead of flattening them, retrieves only as much as a query needs, and separates organization from retrieval so each can be optimized alone. HORMA answers with a file-system-style hierarchy whose summaries index the raw trajectories they came from, navigated by an agent that assembles context on demand.
2. Why current memory methods fail
The paper groups the standard fixes into three families. History hoarding, keeping the entire trace in context, fails because effective context length falls short of the nominal window and the lost in the middle effect makes position matter more than relevance. On LongMemEval, no context limit scores 20.4 against plain truncation's 34.1: feeding the model everything is worse than cutting it off arbitrarily. LoCoMo runs the other way, 55.9 against 32.2, so the effect tracks how long and noisy the history is — context volume is the wrong variable to optimize.
Lossy compression — summarization such as ReSum, folding such as AgentFold and HiAgent — commits irreversibly to a relevance judgment at write time: folding underperforms a naive sliding window on ALFWorld despite preserving reasoning traces, since retained reasoning eats context that could hold actionable information, and ReSum drops to 18.7 on LoCoMo against truncation's 32.2. Flat external memory such as Mem0 and A-MEM scales storage but retrieves by semantic similarity, collapsing temporal and causal structure so it returns facts relevant in subject but stale in time — swapping only the retriever cuts non-reasoning errors such as temporal staleness while reasoning errors stay comparable, which isolates retrieval as the source of the gain.
Beneath all three: most systems jointly optimize construction and retrieval under sparse task reward, which cannot say whether a failure came from bad organization, bad retrieval or bad reasoning. Construction acts slowly with delayed effects; retrieval sits on the inference path and can be trained with a dense evidence-overlap reward. That asymmetry justifies splitting them, and one result seals it: a weak manager cannot be rescued by a much stronger retriever.
3. What a memory layer must do
Working backwards from those failures gives four requirements. None is exotic, and all four are properties of the write path — the part most systems treat as preprocessing and never revisit.
- Survive its own compression. A summary has to be able to give back what it dropped, which means the raw trace stays and the summary points at it.
- Carry time. Timestamps on every note, and relative references resolved into absolute ones while the context needed to resolve them still exists. "Two months ago" is only cheap at write time.
- Be searchable by structure, not only by resemblance. Grouping by entity, by event, by session gives a retriever something to navigate that an embedding index flattens away.
- Represent absence. If nothing in memory says the user owns an iPad, the agent needs to tell "they never mentioned one" apart from "I did not find it." Only the first licenses a confident no.
One system implements all four. Its central move is structural rather than clever: the structure is the part that transfers.
4. The approach: separating two jobs
Section 2 gave the reason to split memory in two; the sparse task reward leaves the signal, in the paper's phrase, "weak and entangled." Two further asymmetries decide which half gets which treatment. Frontier models are already good at semantic abstraction and hierarchical structuring, so decent organization can be prompted out of them, whereas a retrieval policy genuinely improves with training. And there is no clean label for "was this filed well," but an obvious one for retrieval: compare what was pulled against the passages known to contain the answer.
So a memory manager on a strong model organizes, refined by rewriting its own instructions rather than by gradient descent, and a separate retriever, trained with reinforcement learning on a small model, fetches. Both work through the same workspace, and the agent performing the task never sees the raw history at all.
π(at | ot, Ft, q) = Mr(Ct | Ft, q) · Mθ(at | ot, Ct, q; Pmain)
The per-step policy factors in two. Mr, the retriever, picks a context Ct out of the memory workspace Ft. Mθ, the acting agent, chooses its action from that context alone; its weights stay frozen throughout, so every reported difference comes from memory handling rather than a better actor. The manager Mm maintains Ft and never appears on the step path. (from the paper)
5. The architecture: notes that keep their source
The workspace is a directory tree. Not a vector index and not an opaque buffer: files and folders a person could open in a terminal, which both modules manipulate through ordinary shell commands.
The order of operations is what matters. For each step or dialogue turn the manager first archives the raw trajectory into a timestamped directory, and only then writes notes against it, each carrying a compact account of what happened, temporal metadata, and a reference back to its raw trace.
| Directory | Note (summary) | Timestamp | Linked evidence |
|---|---|---|---|
| /people | Alice_profile.md | 2025-05-01 | → traj_0001 |
| /people | Bob_profile.md | 2025-05-03 | → traj_0002 |
| /entities | Meeting_with_Alice.md | 2025-05-05 | → traj_0034 |
| /events | Alice_move_to_seattle.md | 2025-05-06 | → traj_0034 |
| /events | Session_12.md | 2025-05-06 | → traj_0102 |
| /events | Session_13.md | 2025-05-08 | → traj_0150 |
| /raw_logs | traj_0001.jsonl | 2025-05-01 | raw |
| /raw_logs | traj_0002.jsonl | 2025-05-01 | raw |
That link is the whole difference between this and summarization, where compression and deletion are the same act. Separating them costs a pointer and some disk, and turns section 2's irreversible early decision into a reversible one.
Structure, timestamps and provenance links make section 3's middle requirements concrete, and buy something a flat store cannot: a path from any claim to the evidence for it, which is what makes an agent's memory auditable by a human and debuggable at all.
6. Retrieving memory: navigation, not similarity
The retriever turns a question and a directory tree into the smallest context that answers it, by walking the tree rather than ranking embeddings.
| Verb | What it does |
|---|---|
| ls · cd | List and navigate. The agent reads the shape of memory before it reads any contents. |
| grep · cat | Search across files, then open one. Cheap probe first, expensive read second. |
| select | Commits verified content into the retrieved context. |
| done | Ends retrieval once the evidence is sufficient. |
Training it is where section 4's split pays for itself. The reward never looks at whether the final answer was right; it scores the overlap between what was retrieved and the passages already known to hold the evidence.
J(Ct, E) = |Ct ∩ E| / ( |Ct| + |E| − |Ct ∩ E| )
Jaccard similarity between the retrieved context Ct and the ground-truth evidence set E, the union in the denominator written out as |Ct| + |E| − |Ct ∩ E|. The intersection rewards finding the right material; the union penalizes everything extra that came with it, which makes the policy retrieve compactly rather than merely correctly. Because the score ignores what happens downstream, a good retrieval still scores well when the acting agent goes on to reason badly. (from the paper)
The policy is trained with GRPO, which samples a group of attempts per task, normalizes their scores against each other, and reinforces the ones above average. The backbone is Qwen 3.5 4B, small enough to sit on the hot path without anyone thinking about it: learning rate 10⁻⁶, eight rollouts per task, a ten-turn interaction cap, 2,048 tokens per response, four H200 GPUs. It learns navigation habit rather than any particular fact — probe before reading, recover from a failed command, narrow a search that returned too much, stop when the evidence is complete.
7. Improving memory: learning from failure
The manager improves by a different route, and this is the part most teams could adopt tomorrow: it needs no training infrastructure at all.
Run each task twice, once with the full raw history as an unconstrained control and once through managed memory. Most of the time the two agree; the disagreements are the signal, and they fall into two sets with opposite meanings.
| Exogenous · raw history wins | Endogenous · managed memory wins | |
|---|---|---|
| What happened | Something was lost on the way in | Structure defeated a long-context pathology |
| Diagnosis | A fact was never written down, a summary overwrote it, the wrong file was read, a grep missed. The lesson is a gap in the pipeline. | The control hallucinated, or lost the fact in the middle of a hundred thousand tokens. The lesson is a strength to reinforce. |
| Skill it generated | Preserve Object Identity Chains: never abstract "mug 1" into "a mug", and keep a persistent object registry. | Temporal Precision Anchoring: put exact dates and resolved relative times, "'yesterday' relative to Session 6 = July 5, 2023", in every event note. |
An LLM analyst writes up each disagreement and the write-ups fold back into the manager's prompt as new skills — a loop the authors describe as textual gradient descent. Each skill is a structured record: an identifier, a short title, a transferable principle, and a field specifying how the memory writer's schema should change so future runs encode the lesson without being told again. The loop:
- Run the pair on the same task, one with the raw trace, one with managed memory.
- Sort the disagreements into the exogenous and endogenous sets.
- Generate contrastive feedback, with the analyst required to cite trajectory evidence.
- Rewrite the manager's prompt, adding the new skills.
- Repeat. The library grows; it does not replace itself.
On LongMemEval this ran four rounds and produced 63 skills from an empty start, with task performance climbing as the library grew. The output is recognizably domain-shaped: ALFWorld produced Failed-Action Loop Detection, which watches for the simulator's "Nothing happens" reply repeating and treats it as a cue to consult memory for somewhere else to try. The two conversational benchmarks, working independently, converged on the same pair of endogenous skills.
| Skill | Benchmarks | Principle |
|---|---|---|
| Endogenous · advantages of structure over raw history | ||
| Temporal Precision Anchoring | LoCoMo, LongMemEval | Embed exact dates, session timestamps and resolved relative times in every event note, so the model never has to infer "two months ago" from a 100k-token history. |
| Verbatim Quote Preservation | LoCoMo, LongMemEval | Store exact quotes with speaker attribution and turn IDs. Stops paraphrase drift and lets any fact be checked against its source. |
| Entity-Centric File Organization | LongMemEval | File by the entities questions get asked about — people, places, events, devices — so retrieval is targeted instead of a scan for scattered mentions. |
| Goal-Object Spatial Pinning | ALFWorld | The moment a target object is found, write its exact location with high-relevance tagging, before the one useful observation is buried under dozens of failed navigation steps. |
| Exogenous · gaps the pipeline had to close | ||
| Retrieval Path Redundancy | LoCoMo | Index critical facts by person, by date, by topic and by keyword. Single-path storage fails whenever the query arrives framed differently. |
| Encode Negative Evidence Explicitly | LoCoMo, LongMemEval | Record what was not discussed. "Absence-of-evidence is invisible in selective extraction pipelines," so the agent has no honest way to say a thing never came up. |
| Cross-Reference Comparative Facts | LongMemEval | Which happened first, A or B, needs both facts stored with comparable timestamps in retrievable proximity. Siloed notes block the comparison even when both facts were recorded. |
| Retrieval Coverage Diagnostics | LongMemEval | Return metadata on files searched and coverage gaps, so the agent can tell "not in memory" from "in memory but not retrieved" and either widen the search or admit uncertainty. |
8. Experiments: benchmarks and budgets
Three benchmarks span embodied interaction and long conversation. ALFWorld contributes 134 household tasks across six families, on the standard out-of-distribution split. LoCoMo contributes dialogues of nineteen to thirty-two sessions each: 1,089 questions over seven conversations for training the retriever, 519 over three held-out conversations for evaluation, so no evaluation dialogue is seen during training. LongMemEval contributes 367 questions and is never trained on at all, making every number reported on it zero-shot.
Each benchmark runs under a budget set below what an unconstrained agent actually uses, which is the point of the exercise. ALFWorld gets 1,950 and 2,200 tokens against a measured median of 2,172; LoCoMo 10,000 against instances averaging over 20,000; LongMemEval 50,000 against instances averaging over 100,000. Claude Sonnet 4.5 acts in every method compared, so differences trace to memory handling rather than a stronger actor.
The comparisons are section 2's three families: truncation, a sliding window and embedding similarity for cutting the history; ReSum and Acon for compressing it, with context folding and HiAgent added on ALFWorld; A-MEM and Mem0 for moving it outside the context. Alongside them runs one control that matters more than it looks — an ordinary embedding retriever pointed at the structured notes, which isolates what the organization layer contributes from what the navigator does.
9. Results
On conversation, the memory layer shows up clearest in questions about change over time.
| LoCoMo | LongMemEval | |||||
|---|---|---|---|---|---|---|
| Method | single-hop | temporal | overall | knowl. update | temporal | overall |
| No context limit | 78.1 | 66.4 | 55.9 | 37.2 | 14.3 | 20.4 |
| Cut the history | ||||||
| Truncation | 47.8 | 30.8 | 32.2 | 62.8 | 17.3 | 34.1 |
| Sliding window | 39.9 | 47.7 | 31.4 | 19.2 | 15.8 | 16.1 |
| Embedding | 47.5 | 45.8 | 34.8 | 37.2 | 17.3 | 23.2 |
| Compress the history | ||||||
| ReSum | 21.9 | 32.7 | 18.7 | 60.3 | 15.8 | 34.3 |
| Acon | 42.8 | 26.2 | 32.6 | 64.1 | 21.8 | 40.6 |
| Move it outside the context | ||||||
| A-MEM | 39.6 | 35.5 | 30.4 | 64.1 | 32.3 | 51.8 |
| Mem0 | 64.7 | 28.0 | 43.4 | 42.3 | 25.6 | 43.6 |
| Embedding retrieval over structured notes | 57.2 | 33.6 | 37.8 | 44.9 | 18.8 | 30.2 |
| Organize and retrieve | 70.5 | 50.5 | 51.6 | 89.7 | 33.1 | 55.9 |
Less context, better answers
Read the top row of that table against the bottom one. On LongMemEval, where instances run past 100,000 tokens, an agent handed the entire history with no budget at all scores 20.4. Plain truncation under a 50,000-token budget scores 34.1, and a memory layer that organizes and navigates reaches 55.9 on roughly half the tokens. Reading less of the conversation produced better answers than reading all of it, an effect the paper attributes to "implicit filtering of irrelevant context." At that length, dilution and position effects cost more than the discarded material was worth, which is why memory management on long inputs is an accuracy technique that happens also to be cheaper.
Token consumption tells the same story. Every variant answers within 1,000 input tokens per query: 3.07–22.17% of what compared methods spend on LoCoMo, 1.24–16.19% on LongMemEval. The control matters most — plain embedding retrieval over the structured notes lands in the same efficiency band and improves 34.8 → 37.8 on LoCoMo. The compression comes from how memory was written, not from the navigator that reads it.
Two ablations separate the halves: what the self-written skills contribute, and whether the reinforcement learning does anything a prompt could not.
| ALFWorld | LoCoMo | LongMemEval | ||||||
|---|---|---|---|---|---|---|---|---|
| Retriever | Skills | RL | score | calls | score | calls | score | calls |
| Claude Sonnet 4.5 | — | — | 51.5 | 4.47 | 42.2 | 4.98 | 43.6 | 5.47 |
| Claude Sonnet 4.5 | — | 73.9 +22.4 | 4.43 | 51.6 +9.4 | 4.73 | 55.9 +12.3 | 5.26 | |
| Qwen 3.5 4B | — | — | 35.8 | 5.21 | 27.0 | 5.46 | 30.8 | 5.53 |
| Qwen 3.5 4B | — | 40.3 +4.5 | 5.08 | 32.6 +5.6 | 5.33 | 40.6 +9.8 | 5.41 | |
| Qwen 3.5 4B | 64.9 +29.1 | 4.58 | 42.2 +15.2 | 5.12 | 58.0 +27.2 | 5.22 | ||
| Manager | Retriever | F1 | Judge score | Tokens |
|---|---|---|---|---|
| Claude Sonnet 4.5 | Claude Sonnet 4.5 | 21.4 | 51.6 | 308.1 |
| GPT-4.1 | GPT-4.1 | 20.0 | 50.1 | 262.3 |
| GPT-4o mini | Claude Sonnet 4.5 | 24.6 | 55.1 | 287.9 |
| Qwen 3.5 4B | Qwen 3.5 4B | 9.1 | 21.0 | 301.7 |
| Qwen 3.5 4B | Claude Sonnet 4.5 | 9.9 | 23.3 | 287.9 |
Skills help every configuration while reducing retrieval calls, the signature of memory that has become easier to search rather than merely richer. Training the small retriever recovers most of the gap to a frontier model, including on ALFWorld, a domain it never trained on: what transferred was shell technique, not knowledge. On LongMemEval the trained 4B model reaches 58.0 where the frontier retriever reaches 55.9.
Can a strong retriever rescue a weak organizer? No: a 4B manager sits at 21.0 with its own retriever and 23.3 when handed the strongest available. The paper's reading is blunt — "flawed memory organization cannot be compensated for by high-quality retrieval." Capability is needed at write time, once per turn, off the step path; it can be economized at read time, several times per step, on it.
10. Limitations
Four conditions bound how far these results carry.
- The retrieval reward needs labels. The Jaccard score is computed against ground-truth evidence sets taken from the LoCoMo dataset. Benchmarks ship those annotations; deployments do not. The authors name the successor problem themselves: extending the framework to fully online, interaction-driven learning.
- The manager has to be a strong model. The lower half of Table 3. Organization is where capability is non-negotiable, putting a floor under the write path's cost even when the read path is cheap.
- Skill refinement is offline and paired. The contrastive loop needs each task run twice, with an unconstrained raw-history control. That control makes the exogenous and endogenous split possible, and exists at training time only.
- Some question types do not move. On LoCoMo's adversarial split, questions built to be unanswerable, the score is 13.4 where Acon reaches 16.4; on LongMemEval's single-session-preference split, 13.3 where Acon reaches 40.0. Structured retrieval pays where structure exists to exploit: temporal ordering, entity identity, tracked updates.
Drawn from the paper's conclusion, appendix and result tables (from the paper). Every headline figure also runs a single acting model, Claude Sonnet 4.5, across two conversational benchmarks and one embodied simulator.
11. Conclusion
Splitting memory into two jobs is not architectural taste; it is visible in the ablations, where organization skills and a trained retriever contribute independently and each can be measured with the other held fixed. Optimized as one pipeline against one outcome, neither has a signal it can learn from.
Of the two, the write path carries more of the value, so the order in which memory gets written, timestamped and linked back to its source is where the returns are. What navigation adds on top is concentrated where similarity is blind — what changed, what came first, what is current — and that a 4-billion-parameter model does this well after training on dialogue alone suggests what transfers is technique rather than knowledge.
None of the open conditions in section 10 touches the finding underneath all of it: on long histories, the constraint that binds is not how much an agent can be shown, it is how much it can use.
12. References
Primary source. Hsu, H.-L., Kuang, N. L., Liu, B., Yao, Z., and He, Y. (2026). Organize then Retrieve: Hierarchical Memory Navigation for Efficient Agents. Preprint, Duke University and Snowflake AI Research. Posted 10 June 2026, and therefore not yet peer-reviewed. → arXiv:2606.11680v1
What it supplies. Every measurement in this piece comes from that one preprint: the hierarchical workspace and its provenance links, the contrastive skill loop, the evidence-grounded retrieval reward, and the results on all three benchmarks.
How it is marked. Claims and figures taken from the paper are marked from the paper where they appear. The two illustrations redrawn from its own figures are marked redrawn. The argument, and the order it is made in, are ours.
On the name. The system is called HORMA. It appears in the tables above as "organize and retrieve" because this piece is about the approach rather than the name.
Systems named in passing. ALFWorld, LoCoMo, LongMemEval, Mem0, A-MEM, ReSum, Acon and HiAgent, among others, are cited in full in the primary source's own bibliography rather than restated here.