The standard recipe for long-conversation memory is extract-then-store: have an LLM compress the transcript into structured memory records, with people, preferences, and events each filed in their own slot, then retrieve those records on demand. It sounds sensible, and it genuinely saves tokens. Nearly every memory layer shipped in the last two years works this way, and the design is usually defended on context-window grounds. What rarely gets asked is how much the extraction step throws away, and whether the answers you lose are worth the tokens you save.
The paper below asks exactly that, and it does so by removing every other difference between the two designs.
What the paper says
This is a controlled ablation, and it is deliberately narrow. Retrieval uses the same hybrid setup on both sides. Reranking uses the same bge-reranker-v2-m3. The answering model and the grading model are held fixed. The only variable that moves is the storage medium: LLM-extracted structured records on one side, and on the other side the raw conversation with no processing at all, cut into 512-character sliding-window chunks.
The results are lopsided. On LoCoMo (10 real conversations, 699 questions), verbatim chunks score 43.9%, budget-matched structured artifacts score 29.2%, and the best GraphRAG configuration reaches only 13.0%. Adding a one-hop graph on top of verbatim chunks gives 43.1%, which is no improvement. On LongMemEval-S the gap widens to 22.0 points: 67.4% against 45.4%, with a McNemar test at p < 10^-15. Temporal reasoning is where the two diverge most, 50.2% against 31.2%.
The temporal-reasoning split is worth dwelling on, because it points at what extraction actually destroys. Questions about ordering, duration, and change over time depend on how statements sit relative to each other in the transcript. Extraction flattens that. A record capturing "the user prefers X" discards the turn where the user said something different two months earlier, along with the fact that the two statements were two months apart. Raw chunks keep the ordering for free, since it is just the text in the order it was said.
The failure attribution is more persuasive than the headline scores. 69% of artifact failures trace back to the fact never being extracted in the first place. Extraction happens at write time, when the distribution of future questions is unknown, so the model has to guess what matters. The detail an answer needs is often precisely the one that got judged unimportant, because importance at write time is a proxy for salience, not for future usefulness. That is write-time information loss, and no amount of retrieval-side tuning recovers it. It also explains why the GraphRAG numbers are so poor and why the one-hop graph adds nothing on top of verbatim chunks: a graph built over already-lossy extractions inherits the loss, and a graph layered over text that already contains the answer has nothing left to contribute.
Cost was supposed to be extraction's home turf, and the argument does not hold there either. Artifacts are 5.1x shorter than the raw text, so a single answer really is cheaper to produce. But normalized per thousand correct answers, verbatim chunks come to $12.5 and structured artifacts to $14.9, and that figure still excludes the upfront cost of running extraction at all. The token savings get eaten by the lower hit rate.
How strong is the evidence
As ablation design goes, this one is clean. Single variable, fixed pipeline, a significance test actually reported, and costs converted into a comparable unit rather than left as raw totals. Budget-matching the artifacts matters too: without it, the comparison would collapse into "more context wins," which nobody needed a paper to learn. The direction replicates across two benchmarks, and the effect sizes are not marginal.
There is one contradiction the author chose to publish rather than bury: on a synthetic benchmark the result goes the other way. The plausible reading is that synthetic data built for extraction-based methods is validating itself, because the question-construction procedure already assumes answers live in extractable entities and relations. That explanation is reasonable, and it doubles as a warning. When the benchmark and the method share an ancestor, the evaluation measures fit rather than capability. That lesson matters more to readers of benchmarks in general than it does to this paper.
The scope of the conclusion is worth stating precisely. What gets compared here is "extraction as a replacement for the raw text" versus "the raw text," not "extraction" versus "no extraction." Structured records remain useful when you need to aggregate across sessions, or when a human has to read the result. The conclusion should be read as: structured memory should augment the raw text, not stand in for it.
What it means in practice
The knowledge base and session memory in FIM One follow one rule. The raw text is always present, and the structured index only handles navigation. It is never the sole source of truth. The price of that rule is higher storage and retrieval cost, and the cost analysis above suggests the price is worth paying. It also means summaries and entity records stay disposable: if the extraction schema changes, the index gets rebuilt from text that was never discarded.
For teams building RAG systems, there is one architectural decision you can copy directly and one question you have to answer yourself. The decision: do not let a distilled representation become the only retrievable object. The question: every time your system decides at write time that this item matters and that one does not, it is placing a bet with the question distribution still unknown. Any pruning that can wait until query time should wait.