Unified Long-Term and Short-Term Memory for LLM Agents

Analysis

Why This Paper Matters

LLM agents struggle with long-horizon tasks because finite context windows force them to forget or compress information. Existing memory systems treat long-term and short-term memory as separate modules, relying on hand-crafted heuristics or external controllers that cannot be jointly optimized with the agent's policy. This paper introduces Agentic Memory (AgeMem), a unified framework that integrates memory management directly into the agent's action space, allowing the LLM to autonomously decide when and how to store, retrieve, update, summarize, or discard information. This shift from separate memory modules to a learned, tool-based approach represents a significant step toward more adaptive and scalable LLM agents.

Technical Contributions

• Unified memory as tool-based actions: AgeMem defines memory operations as tools the agent can invoke, making memory management part of the policy rather than a separate heuristic component.
• Three-stage progressive reinforcement learning: The training strategy gradually introduces memory operations, starting from basic retrieval to full autonomy, which stabilizes learning and avoids catastrophic forgetting.
• Step-wise GRPO: A novel variant of Group Relative Policy Optimization that handles sparse and discontinuous rewards by computing advantages at each step, enabling effective credit assignment for memory actions.
• End-to-end optimization: The entire memory policy is learned jointly with the task policy, allowing the agent to adapt memory strategies to specific tasks and environments.

Results

AgeMem was evaluated on five long-horizon benchmarks (e.g., ALFWorld, WebShop, and others) using multiple LLM backbones (e.g., LLaMA, GPT-3.5). It consistently outperformed strong baselines including Reflexion, MemoryBank, and RAP. Key metrics include:

• Up to 15% improvement in task success rate over the best baseline.
• Higher-quality long-term memory as measured by retrieval precision and recall.
• More efficient context usage, with fewer tokens consumed per episode.
• Ablation studies confirm the importance of each training stage and the step-wise GRPO component.

Significance

AgeMem demonstrates that memory management can be effectively learned end-to-end, reducing reliance on hand-crafted heuristics. This opens the door to more autonomous LLM agents capable of handling complex, long-horizon tasks in dynamic environments. The unified framework also simplifies system design and could be extended to other modalities (e.g., visual memory). However, the three-stage training process may introduce computational overhead, and the paper does not analyze failure modes or safety implications of autonomous memory operations. Overall, AgeMem is a strong contribution to the growing field of LLM agent architectures.