Artificial Intelligence1 min read

Memorilla: Latent Semantic Memory for LLMs

By · Published by Everything Blog

In short

The architecture functions by converting text sources like chat histories into reusable embeddings, which are later distilled into query-dependent memory tokens inside the model's input space. Evaluations across multiple benchmarks proved it can outperform established alternatives like Mem0 and MemoRAG. Additionally, the system curbed repetitive agent wandering in interactive environments while cutting down the context token count drastically.

Key points

  • Condensing external documents into compact memory tokens lowers computational overhead co…: Condensing external documents into compact memory tokens lowers computational overhead compared to processing entire text contexts.
  • The system prevents repetitive exploration behaviors and enhances task completion during…: The system prevents repetitive exploration behaviors and enhances task completion during interactive agent simulations.
  • Pre-encoding source documents once provides frozen models with a persistent and efficient…: Pre-encoding source documents once provides frozen models with a persistent and efficient semantic memory mechanism.

Sources checked by our analysis pipeline: 1

Computer Science > Machine Learning

[Submitted on 15 Oct 2026 (v1), last revised 3 Nov 2026 (this version, v3)]

Title:Memorilla: Latent Semantic Memory for LLMs

View PDF HTML (experimental)Abstract:Memory enables large language models (LLMs) to use information beyond their parameters, such as past user conversations, long documents, or an agent's earlier actions. For a frozen LLM, access to this accumulated knowledge is limited to what a memory system places in its context window. However, gathering and combining relevant information needed for each query while keeping the model's context short remains challenging for existing systems. Here we introduce Memorilla, a memory system that gathers information from an entire document collection into a fixed number of query-dependent 'memory tokens' for a frozen LLM. Memorilla encodes documents (such as user chats, book chapters) once into reusable embeddings, then learns to retrieve and place query-specific information in the LLM's input embedding space. Using only 16 memory tokens, Memorilla achieves the best performance against baselines including Mem0 and MemoRAG on five of eight benchmarks spanning personalization, long-document and knowledge-intensive question answering. Notably, on FactKG, it…

Artificial Intelligence

More in Artificial Intelligence