Five AI Agent Harnesses Took the Same Memory Test. The Results Weren't Even Close.
The New Stack, Thursday, October 22nd, 2026
Letta's FrontierMemory benchmark tests whether agents learn from past work; harness choice swung scores widely.
Letta's new FrontierMemory benchmark measures whether agents can turn transcripts of earlier successes and failures into memories that improve later tasks, rather than just recalling facts.
Across 15 tasks, Letta Dream scored 73.3%, Prime Agent 60%, OpenAI Agents SDK 40%, Claude Managed Agents 33.3% and Hermes Agent 26.7%.
With the same model, Dream beat the OpenAI SDK 60% to 40%, suggesting the harness matters as much as the model.
The article notes the benchmark comes from a vendor whose product won, and warns that without checks on what gets saved, agents can learn and repeat mistakes.