Read-Time Memory Curation (JitMem)

Sources#

• Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents

Summary#

Agent-memory systems that reuse past experience mostly curate at write time. When a task finishes, its trajectory is distilled into a fixed artifact: a Reflexion reflection, an ExpeL insight, an Agent Workflow Memory workflow, a Voyager or SkillOS skill, a ReasoningBank strategy. Later tasks retrieve that artifact by similarity. Just-in-Time Memory (Yefan Zhou, Yang Li, Zeyu Leo Liu, Semih Yavuz & Shafiq Joty, Salesforce AI Research, arXiv 2609.27334, 2026-09-23, empirical) names two costs of that design. Information discarded at write time cannot be recovered by a later task that needs it. And one query-independent artifact has to serve every future task, although one trajectory holds many lessons and which one matters depends on a task not yet seen.

The fix is to defer curation to read time. The memory bank stores complete raw trajectories, gated only by the executor judging its own run successful. At each new task, BM25 over task descriptions retrieves k=3 trajectories. A curator reads them together with the current task and writes a compact briefing (the payload), which is prepended to a frozen executor's prompt and then thrown away. Only the curator is trained.

Four results carry the page:

1. Read-time beats write-time with the same curator model and no training, in most but not all cells. With Gemini-2.5-Pro curating for a Gemini-2.5-Pro executor on WebShop, untrained JitMem reaches 61.0 SR against 41.0 for SkillOS-gemini and 40.2 for ReasoningBank.
2. Raw storage carries information that write-time distillation loses. Replace raw traces with ReasoningBank-style distilled items and the untrained curator drops 6.8–8.2 SR on WebShop and 1.7–2.9 on ALFWorld, across all three executors.
3. The training signal gets simpler. The payload is consumed on the task it was written for, so the curator's reward is that task's success, with zero delay. GRPO on task reward alone beats RL-trained SkillOS, which needs grouped task streams and a judge-model content reward, by 16.2 SR on ALFWorld and 16.3 on WebShop with a Qwen3-8B executor.
4. Not all of the trained gain is memory. With retrieval forced empty, the trained curator still beats the untrained curator with retrieval in 3 of 6 executor×benchmark cells. On WebShop with Qwen3-8B it beats RL-trained SkillOS (17.6 vs 16.5 SR) without reading any memory at all. The paper's own sentence on this ablation overstates it (see below).

The mechanism#

Four components: a bank M of raw trajectories, an untrained BM25 retriever over task descriptions only, a curator π_φ, and a frozen executor π_L. Each task runs retrieve → curate → execute → update. The update appends the trajectory only if the executor, acting as LLM judge, says the task succeeded. Ground-truth labels never enter the bank. The payload is ephemeral: the bank holds trajectories, never payloads, so the stored layer is never rewritten by curation.

Training uses GRPO with group size 8, batch 32, lr 1e-6, 100 steps, and no standard-deviation normalization (the Liu et al. 2025 "Dr. GRPO" variant; see Group Relative Policy Optimization (GRPO)). The policy is the curator, the frozen executor is the environment, and reward is the benchmark's native score. The training bank is built once from base-executor runs filtered by ground truth and held fixed, so reward reflects payload quality rather than which trajectories happened to be available. At test time the bank starts empty and grows in batches of 10 (5 on τ²-bench). One training run takes about 21 h on ALFWorld and 27 h on WebShop on 8×H200.

The credit-assignment argument is the design's real novelty. A write-time curator's storage decision at step s is graded only when some later task t > s retrieves it. SkillOS manufactures that signal by grouping related tasks, and its own ablations name grouping as a major contributor. Moving the decision next to its reward removes the delayed-credit problem rather than solving it. That is an architectural alternative to the per-turn credit machinery on Turn-Level Credit Assignment: no return is split, because the gap between action and reward is zero.

Results, verified against the PDF#

Tables 1–5 and 9 parsed clean. Every number below was re-read from pdftotext -layout on the local PDF, and the ablation ranges in the prose were recomputed from Table 9. Mean ± std over 3 task orderings (4 on τ²-bench). All main results use k=3.

Learned read-time vs learned write-time (Table 1). RL-trained Qwen3-8B curator in both cases:

Three things in this table the headline does not say. (a) The untrained read-time curator loses on the weakest cell. On WebShop with Qwen3-8B, JitMem-base's 11.7 SR is below SkillOS-base (13.6), MemP (12.0) and SkillOS-gemini (13.2), and its score (32.5) is below no memory (33.3). The "read-time generally outperforms" claim holds for the stronger executors, not the 8B one. (b) A prompted frontier curator beats the trained 8B curator on WebShop. JitMem-gemini reaches 61.0 SR against the trained curator's 50.5 on the same executor. The trained curator's advantage is cost (an 8B model), not ceiling. (c) RL-trained SkillOS was never run with the GPT-5.4 executor. In that block JitMem's "+8.8 / +10.9 over the strongest baseline" is measured against untrained write-time methods only.

τ²-bench (Table 2, GPT-5.4 executor, training-free variants only). There is no standard training split, so no curator was trained. JitMem-gpt reaches 75.6 micro-average SR against 71.7 for ReasoningBank-gpt (+3.9). The whole gain is Telecom: 72.6 vs 61.6, with std 4.5 and 8.4. On Airline, JitMem-gpt (63.5) is below no memory (65.0), and the paper concedes that on Airline and Retail no memory method beats no memory beyond variance. Its reading: read-time curation pays where tasks need synthesized procedural guidance (multi-step policy verification), not fact lookup.

Executor transfer (Table 3). A curator trained with a Qwen3-8B executor and run with GPT-5.4 scores 86.7 on ALFWorld. One trained directly with GPT-5.4 scores 88.1, a gap of 1.4. Over the untrained curator, the transferred one adds +6.2 / +7.4 on ALFWorld (Gemini / GPT-5.4) and +6.1 on WebShop for both. The paper sets this against MemHarness, a concurrent read-time system that trains one policy to both curate and execute and so cannot transfer across executors.

Tokens and steps (Table 4, ALFWorld, GPT-5.4 executor only). Executor input tokens: no memory 9.0K, ReasoningBank 19.7K, SkillOS-base 22.4K, JitMem-base 10.9K, JitMem 9.8K. Steps fall from 17.8 to 11.6. The headline "50.3–56.3% fewer input tokens, 28.4–31.4% fewer steps" is measured against the two write-time methods. The count is executor-side only. The curator call, which reads three full raw trajectories on every task, is not in it, and neither are ReasoningBank's write-time distillation calls. The limitations paragraph says only that the curator "adds an extra LLM call per task". So the compactness result is a claim about what the executor reads, not about total spend. That is the same accounting gap Cache-Aware Agent-Loop Cost (TurnCost) warns about for executor-only token counts.

The ablations (§4.2, Table 9)#

Untrained JitMem-base, each design choice removed in turn, SR drop across the three executors:

After RL, removing task conditioning costs up to 11.4 on ALFWorld and 10.4 on WebShop, wider than before training. That is the paper's evidence that RL learns to use the task signal rather than to compress better. The ALFWorld drops for the untrained curator mostly sit inside one standard deviation (1.5–3.6). The WebShop raw-storage drop is the robust number: on Qwen3-8B, distilled-only storage falls to 4.9 SR, half the no-memory 9.8. Staged bank refresh adds at most +2.8 SR (Qwen3-8B, WebShop). Warm-starting the test bank with 100 training trajectories moves SR by at most 1.3, within std. Raising k from 3 to 5 changes WebShop SR by under 2 points.

Where the paper's own ablation sentence overstates. §4.2 says that with retrieval forced empty, the trained curator "degrades to or even below the untrained JitMem-base across all executors", which would confirm that RL learns to distill retrieved experience. Table 9 shows otherwise:

Retrieval clearly matters, since removing it costs up to 14.8 / 15.2 SR. But in half the cells, an RL-trained curator with nothing to read writes better briefings than an untrained one reading three relevant past runs. Figure 4's qualitative finding says the same thing from the other side: RL "induces environment-specific procedural semantics" that the curation prompt never specified. Part of what the curator learns is the procedure itself, held in its weights. That is compilation, done by gradient rather than by a write-time distiller, and it is the reason for this page's first open question.

How it bears on the wiki's compile-vs-retrieve and curate-vs-store arguments#

It is not the retrieval side of LLM-as-Compiler Knowledge Base's axis. JitMem never hands the executor raw retrieved text. The executor always reads a synthesized payload, so the system compiles per query rather than once. Its target is narrower: distill once at write time, then discard the source. This vault never does that. raw/ is immutable, the compiled page sits beside it, and /query re-synthesizes from both at read time. The vault is already the hybrid JitMem's ablations never test: a write-time distillation kept alongside the raw store, with a task-conditioned read on top. The raw-storage ablation measures distilled-instead-of-raw, which strikes a design neither this vault nor JitMem uses.

It contradicts Muscle Memory's P1 on its own terms. That paper (Google Cloud, empirical) assigns retrieval to "one-off facts whose interpretation depends on the live request" and compilation to "recurring procedures whose interpretation is itself the part that should be cached." JitMem finds the opposite split. Read-time, task-conditioned synthesis helps most on procedural tasks (ALFWorld, WebShop, τ²-Telecom's multi-step policy checks) and not at all on Airline and Retail lookups. Weighting: both are empirical. JitMem has the controlled arms (write-time baselines with the same curator model, and a distilled-storage ablation), while Muscle Memory benchmarks only against no memory, so on the narrow question of text distilled at write time versus text synthesized at read time, JitMem is the stronger evidence. It does not refute Muscle Memory's actual artifact. Muscle Memory compiles to tested executable specialists, and no JitMem arm includes one. The finding above, that the trained curator caches procedure in its weights, also means JitMem ends up compiling the procedural part somewhere. The narrowed conflict is recorded here, and as one sentence on LLM-as-Compiler Knowledge Base.

It partially answers Knowledge-Centric Self-Improvement's curation-vs-storage question, and splits it. The question asks whether that paper's forums and distillation beat a raw attempt table. JitMem shows that "curation" bundles two choices that come apart: what is stored (raw vs distilled) and what the reader receives (raw vs synthesized for its task). On the first, raw wins, robustly on WebShop and weakly on ALFWorld. The second is untested: no arm feeds the three retrieved trajectories straight to the executor, which is Synapse's design, named in Related Work and not run. That is the same "raw table only" arm the knowledge-centric question asks for, so it is not repeated as a separate question here. That paper's own knowledge-transfer adapter, which caps every field at 0–3 items selected against the recipient task (see LLM-as-Compiler Knowledge Base), is already read-time task-conditioned selection. Its authors added it after observing that delivering a fixed quantity made recipient memory "overly noisy or detrimental."

It sits upstream of Crystallizing Agent Work into Workflows. Malik's lifecycle commits agent behavior to a fixed playbook, which is write-time curation taken all the way to deterministic code. JitMem's premature-commitment argument applies to it. But crystallization defers the commitment until a playbook has repeatedly validated, and it demotes automatically on regression, so it fixes a behavior only once its future use is predictable. JitMem's argument is strongest where that predictability is absent: heterogeneous task streams in which one trace teaches different lessons to different tasks (Figure 3's hot-potato vs newspaper-on-sofa pair).

The memory-safety reading#

Two design choices matter for Memory and Context Poisoning, and the paper discusses neither. Admission is self-judged. A trajectory enters the bank permanently when the executor, the same model that produced it, judges it successful. No ground truth, provenance or second model is involved, and judge precision is never reported. Raw storage keeps every tool output verbatim, for as long as the bank exists. Write-time distillation at least passes stored content through a summarizer once. A raw bank keeps any injected tool output intact and has the curator restate it, in the curator's own voice, inside the executor's prompt on every later task that retrieves it. That is the self-summarization laundering route Non-Malleable Memory Authority (TMA-NM) proves content- and lineage-based defenses cannot see. Here it runs on every read rather than once at write. The paper's argument for raw storage, that nothing is irreversibly lost, is from the attacker's side also the reason nothing is ever scrubbed.

Limitations worth carrying#

• Inherited baseline rows. Table 1's no-memory rows for the Qwen3-8B and Gemini executors are SkillOS's reported numbers (Table 6), not runs from this paper. The authors' own Qwen3-8B reproduction reached 42.1 (thinking) or 34.5 (non-thinking) on ALFWorld against the reported 47.9. Table 6's caption says that "in all cases" reproduction came in at or below the reported value. That is false for WebShop score in non-thinking mode, the mode adopted for WebShop (36.4 vs 33.3). The paper does not say whether the ReasoningBank, MemP and SkillOS rows were re-run or copied.
• Prompt adjustments to the baseline executor. The Qwen3-8B WebShop executor prompt gained a search-guidance paragraph, calibrated so that no-memory SR stayed at or below the reported 9.8. The same prompt is used for every method, so this shifts all rows, but it is a tuned harness.
• One distiller in the ablation. "Write-time distillation discards information" is measured with one ReasoningBank-style prompt capped at 3 items per trajectory, which is aggressive compression. Richer write-time artifacts (executable skills, longer workflows) were not ablated.
• Scope. There are three benchmarks, two of them simulated household and shopping environments. τ²-bench is training-free only. Payload format is hand-designed per benchmark. The retriever is BM25 over task descriptions only, which the authors flag as the likely bottleneck as the bank grows, and the bank never grew past one test sequence. No COI beyond the authors' own method; the baselines come from other labs (ReasoningBank and SkillOS from Ouyang et al.).

Connections#

• Knowledge-Centric Self-Improvement — the curation-vs-storage question this source was scouted for. JitMem splits "curation" into a storage axis, where raw beats distilled, and a delivery axis, which is unmeasured, and runs the first of the two arms that question names
• LLM-as-Compiler Knowledge Base — the compile/retrieve axis. JitMem is compile-per-query, not retrieval, and its raw-vs-distilled ablation strikes a design (distill and discard) that this vault never uses. It also narrows Muscle Memory's P1, as described above
• Crystallizing Agent Work into Workflows — the write-time end of the spectrum, made safe by deferring commitment until track record makes future use predictable. JitMem's premature-commitment argument targets the regime where it cannot be predicted
• Turn-Level Credit Assignment — the opposite fix for delayed reward: split a terminal return across turns there, or move the decision next to its reward here
• Group Relative Policy Optimization (GRPO) — the optimizer, used with the policy as a memory curator and a frozen executor as the environment, and with the std normalization dropped
• Memory and Context Poisoning — a self-judged admission gate in front of a store that keeps raw tool outputs forever
• Non-Malleable Memory Authority (TMA-NM) — the laundering route through self-summarization, which here runs on every read
• Context Lifecycle Management — the in-session counterpart. That page curates the live context window within one task, while JitMem curates persistent episodic memory across tasks, and the paper places Sculptor and ContextCurator, both RL-trained in-session context curators, on that side of the line
• Harness Activation and Adherence — the consumer-side gate the payload has to clear. JitMem measures only end-to-end SR and step counts, never whether the executor followed the briefing. The τ²-bench executor prompt explicitly invites it to decide "if you want to use past experiences or not"
• Cache-Aware Agent-Loop Cost (TurnCost) — why executor-only token counts understate a design that adds an LLM call per task

Open Questions#

• How much of a trained read-time curator's gain is memory and how much is procedure in its weights? With retrieval emptied, the trained curator beats the untrained one with retrieval in 3 of 6 cells. The settler is a curator GRPO-trained with retrieval disabled throughout, at the same budget, compared against JitMem on ALFWorld and WebShop. If it closes most of the gap, the "memory" result is mostly a learned hint generator.
• Does a write-time distillation kept alongside the raw trajectories beat raw alone when a read-time curator sees both? That is this vault's own raw-plus-compiled design. JitMem ablates only distilled-instead-of-raw. The settler is a hybrid arm on JitMem's harness (same BM25, k=3) in which the curator receives each trace together with its ReasoningBank items.

Sources#

• Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents — Yefan Zhou, Yang Li (equal contribution), Zeyu Leo Liu, Semih Yavuz & Shafiq Joty (Salesforce AI Research), Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents, arXiv 2609.27334, 2026-09-23, 25pp, empirical (the raw is tagged case-study; tier corrected at compile, since the paper reports controlled multi-run results with ablations). Cited: §3 the four-component pipeline, the judge-gated bank and the GRPO objective; §4.1 Tables 1–4; §4.2 the ablation prose, Table 5 and Table 9 (Appendix B); §4.3 Figures 3–4; Appendix A Table 6, the reproduction paragraph and the hyperparameters; §5 the limitations. Parse notes: Tables 1–5, 8 and 9 parsed clean (docling renders mean ± std as "47. 9 1. 2") and were reconciled digit for digit against pdftotext -layout on the local PDF. Table 7 (GRPO hyperparameters) is collapsed: the optimization block is welded into one cell, and three orphan split-row fragments ("32", "32", ". 2, 0. 2)") are scattered. It was read from the PDF and is cited here only for values the §4 prose also states. Two canary misses (50.3%–56.3%, 28.4%–31.4%) are en-dash formatting; the text is present. Every SR delta in the ablation section was recomputed from Table 9, and the "degrades to or even below JitMem-base across all executors" claim fails in 3 of 6 cells