Benchmarks
Hebby is King.
PrecisionMemBench checks whether an AI memory returns exactly the right fact, not just anything. Hebby solves the most precision tasks of all systems that did not design the test themselves.
PrecisionMemBench
- Published as a paper on arXiv
- Open dataset and leaderboard on Hugging Face
- 89 test cases, 13 memory systems
- A standard part of a public benchmark suite for agent memory, next to LongMemEval and LoCoMo
- Adopted by other memory vendors into their own public test suites
Single-turn results
Results per single question, in the same form as the original table.
| Provider | Active passes | Total passes | Mean precision | Mean recall | Retrieval p50 (ms) | Ingestion total (s) | Evaluated (UTC) |
|---|---|---|---|---|---|---|---|
tenure (designed the test ;) ) | 43/43 | 77/77 | 1 | 1 | 9.25 | 0.80 | 09-16-2026 |
hebby | 28/431 | 37/771 | 0.531 | 0.98Top 3 | 207.00 | 17.22 | 11-03-2026 |
open-knowledge-format | 18/43 | 36/77 | 0.47 | 0.91 | 3534.53 | 0.00 | --- |
atomicmemory | 5/43 | 16/77 | 0.23 | 0.54 | 163.85 | 664.80 | 09-15-2026 |
supermemory | 2/43 | 15/77 | 0.18 | 0.82 | 45.77 | 1.70 | 09-17-2026 |
yourmemory | 0/43 | 21/77 | 0.17 | 0.88 | 464.08 | 24.80 | 09-12-2026 |
vector | 0/43 | 11/77 | 0.09 | 1.00 | 71.87 | --- | --- |
gbrain | 0/43 | 11/77 | 0.07 | 1 | 1455.98 | 52.60 | 09-13-2026 |
hindsight | 0/43 | 11/77 | 0.07 | 1 | 610.36 | 173.00 | 09-12-2026 |
zep | 0/43 | 9/77 | 0.06 | 0.66 | 146.41 | 485.80 | 09-15-2026 |
agentmemory | 0/43 | 9/77 | 0.06 | 0.99 | 148.30 | 8.50 | 09-15-2026 |
cognee | 0/43 | 9/77 | 0.06 | 0.98 | 293.18 | 495.00 | 09-15-2026 |
mem0 | 0/43 | 9/77 | 0.06 | 0.99 | 61.15 | 132.80 | 09-15-2026 |
a-mem | 0/43 | 7/77 | 0.05 | 0.99 | 12.44 | 46.80 | 09-16-2026 |
Pass types
The counts below represent the 77 single-turn cases.
| Provider | Active retrieval | Structural | Trivially empty |
|---|---|---|---|
tenure (designed the test ;) ) | 43 | 25 | 9 |
hebby | 281 | 6 | 3 |
open-knowledge-format | 18 | 13 | 5 |
atomicmemory | 5 | 8 | 3 |
supermemory | 2 | 10 | 3 |
a-mem | 0 | 5 | 2 |
agentmemory | 0 | 6 | 3 |
cognee | 0 | 6 | 3 |
gbrain | 0 | 8 | 3 |
hindsight | 0 | 8 | 3 |
mem0 | 0 | 6 | 3 |
vector | 0 | 8 | 3 |
yourmemory | 0 | 15 | 6 |
zep | 0 | 6 | 3 |
Session results
The session evaluation contains 12 scored turn-level assertions across two sessions. The drift score is the fraction of retrieved beliefs originating from off-topic turns. A score of 0 means that no returned belief originated from an off-topic turn.
| Provider | Turns passed | Pass rate | Mean drift | Mean precision | Session p50 (ms) | Evaluated (UTC) |
|---|---|---|---|---|---|---|
tenure (designed the test ;) ) | 12/12 | 1.00 | 0.0000 | 1.0000 | 49.74 | 09-16-2026 |
hebby | 2/121 | 0.171 | 0.54352 | 0.45652 | 968.42 | 11-03-2026 |
open-knowledge-format | 2/12 | 0.17 | 0.2153 | 0.5694 | 3349.45 | --- |
supermemory | 1/12 | 0.08 | 0.6944 | 0.1668 | 164.38 | 09-17-2026 |
yourmemory | 0/12 | 0.00 | 0.7444 | 0.1879 | 568.08 | 09-12-2026 |
atomicmemory | 0/12 | 0.00 | 0.8611 | 0.1389 | 403.97 | 09-15-2026 |
vector | 0/12 | 0.00 | 0.9142 | 0.0858 | 256.75 | --- |
a-mem | 0/12 | 0.00 | 0.9259 | 0.0741 | 22.36 | 09-14-2026 |
gbrain | 0/12 | 0.00 | 0.9259 | 0.0741 | 2001.14 | 09-14-2026 |
hindsight | 0/12 | 0.00 | 0.9259 | 0.0741 | 1217.50 | 09-12-2026 |
mem0 | 0/12 | 0.00 | 0.9259 | 0.0741 | 259.26 | 09-15-2026 |
cognee | 0/12 | 0.00 | 0.9295 | 0.0705 | 483.71 | 09-15-2026 |
zep | 0/12 | 0.00 | 0.9320 | 0.0680 | 357.21 | 09-15-2026 |
agentmemory | 0/12 | 0.00 | 0.9398 | 0.0602 | 339.12 | 09-15-2026 |
As of 2026-11-03, Hebby with factory settings, without reranker. Comparison values: published PrecisionMemBench results as of September 2026.
Early Access
Don’t worry, be Hebby.
An agent that remembers. Hebby is now rolling out to our first users. Tell us briefly what you work with.
