Side-Path Parametric Memory for LLM Agents: Exactly Erasable Slots, Read Without Context
Side-Path Parametric Memory for LLM Agents: Exactly Erasable Slots, Read Without Context
Haotian Cheng1
作者信息
- 1. Nanjing University of Aeronautics and Astronautics
- 折叠
摘要
Long-term memory in LLM agents lives in the context window, where it competes with the tokens needed to reason and is indistinguishable from the current working set. We study an alternative: a capacity-bounded side path of low-rank slots on a frozen backbone, written by gradient steps, read through the forward pass, and forgotten by erasing a rank block. On a 0.6B model, writes touch only their own slot; erasure is bit-identical to a never-written slot across 30/30 evictions and a process restart; and the binding constraint turns out to be read-time composition, since summing the slots destroys recall (0/40) while selecting one by query-key similarity recovers it (35/40 against an oracle of 38/40), with the residual gap equal to the routing error. On real benchmark questions one retrieved fact answers 0.033 of them and all of them 0.679. A label-free self-check reaches full-write recall using 5 writes instead of 18, and 0.54 nats of backbone drift instead of 2.59. Two hypotheses fail, and we report them: a likelihood-threshold surprise criterion measures wording, not knowledge, and parametric memory does not reduce context inertia. On 30 LongMemEval single-session questions with the history absent, the written arm reproduces 1.000 of the reference answers at no extra prompt-token cost, while context delivery answers 0.867 to 0.967 and pays 117 to 1688 tokens. At equal item-gradient-steps the bank recalls 1.000 of those thirty memories where full fine-tuning reaches 0.133 and one equal-rank adapter 0.000, and a model-free change of retrieval key lifts 1.7B routing from 0.719 to 1.000.
Abstract
Long-term memory in LLM agents lives in the context window, where it competes with the tokens needed to reason and is indistinguishable from the current working set. We study an alternative: a capacity-bounded side path of low-rank slots on a frozen backbone, written by gradient steps, read through the forward pass, and forgotten by erasing a rank block. On a 0.6B model, writes touch only their own slot; erasure is bit-identical to a never-written slot across 30/30 evictions and a process restart; and the binding constraint turns out to be read-time composition, since summing the slots destroys recall (0/40) while selecting one by query-key similarity recovers it (35/40 against an oracle of 38/40), with the residual gap equal to the routing error. On real benchmark questions one retrieved fact answers 0.033 of them and all of them 0.679. A label-free self-check reaches full-write recall using 5 writes instead of 18, and 0.54 nats of backbone drift instead of 2.59. Two hypotheses fail, and we report them: a likelihood-threshold surprise criterion measures wording, not knowledge, and parametric memory does not reduce context inertia. On 30 LongMemEval single-session questions with the history absent, the written arm reproduces 1.000 of the reference answers at no extra prompt-token cost, while context delivery answers 0.867 to 0.967 and pays 117 to 1688 tokens. At equal item-gradient-steps the bank recalls 1.000 of those thirty memories where full fine-tuning reaches 0.133 and one equal-rank adapter 0.000, and a model-free change of retrieval key lifts 1.7B routing from 0.719 to 1.000.关键词
parametric memory/LLM agents/low-rank adapters/memory slots/exact erasure/read-time composition/context costKey words
parametric memory/LLM agents/low-rank adapters/memory slots/exact erasure/read-time composition/context cost引用本文复制引用
Haotian Cheng.Side-Path Parametric Memory for LLM Agents: Exactly Erasable Slots, Read Without Context[EB/OL].(2026-10-10)[2026-10-14].https://chinaxiv.org/abs/202610.00050.学科分类
计算技术、计算机技术
