Contents Zeta on BEAM-10M
Research·Memory Systems·November 18, 2026
Zeta, an experimental memory model
BEAM-10M, the reader's effort, and the models since July
- Published
- Nov 18, 2026
- Area
- Memory Systems
- Model
- Zeta, experimental
- Benchmark
- BEAM-10M, BEAM-1M
- Reader
- Claude Opus 5.5
- Judge
- gpt-4.1-mini
Since July we have measured our memory engines in five reports. Each left a question open, and the next model began there. This note reports what we measured with Zeta, an experimental model that starts from Scroll 2: its score on BEAM-10M, what the reader's effort buys, and the models since July on the same store.
Zeta is an experimental model that starts from Scroll 2 and differs from it in several ways. One of those differences can be switched off on its own, and this note gives the runs with it switched off beside the others, marked “off”. The engine is proprietary: this note reports what we measured, and does not describe what was changed or how it works. Zeta is not a released product.
What this note measured
01
Zeta on BEAM-10M
How does Zeta score?
BEAM-10M asks 200 questions over ten conversations of about ten million tokens each. Zeta scored 70.39 and 70.21, the same settings run twice. Scroll 2, with the same reader (Claude Opus 5.5 at maximum effort) and the same judge (gpt-4.1-mini), scored 65.48. The difference per question is +4.8 points (95% interval +0.7 to +8.9). The reader received 5,444 tokens of memory per question on average, fewer than the 5,794 it received from Scroll 2, and it was shown 66% of the evidence the benchmark marks, against 47%. Zeta's configuration was chosen on this benchmark.
A third run
After these two runs one more change was made to Zeta. The third run, made after it, scores 72.52: +7.0 points above Scroll 2 (95% interval +2.6 to +11.5) and +2.2 above its two earlier runs (−1.2 to +5.7). The difference is concentrated in the 60 questions of the three abilities this note groups as asking about time (event ordering, summarization and temporal reasoning), which go from 53.5 to 60.7 (+7.2, 95% interval −0.3 to +14.8); the other 140 questions stay where they were (77.5, then 77.6). The runs with the part off and the runs at lower effort, in the parts below, were made before this change.
02
The reader's effort
What if the reader thinks less?
Every run above used the reader at maximum effort. With the same Zeta, we lowered only the reader's effort and ran four more times, once at each lower level. At low, medium, high and xhigh effort the scores are 64.64, 64.58, 64.77 and 63.40. The two runs at maximum effort are +6.0 points above the mean of these four (95% interval +2.6 to +9.3). Raising the effort did not raise the score until the maximum. Each of these runs was made once.
At medium effort the reader searches again about once: it calls the tools 1.25 times per question and receives 1,386 tokens of memory. It gains little from that search: it is shown 53% of the evidence the benchmark marks, and the first list already holds about 51% of it. The score at medium effort differs from Scroll 2's at maximum effort by −0.9 (95% interval −5.5 to +3.7). The reader costs $0.035 per question against $0.332, and takes 10 seconds against 96. At low effort it calls the tools 0.69 times, reads 1,092 tokens of memory and scores the same, for $0.029 per question.
What does maximum effort buy?
What maximum effort buys is on the questions about time. On the 140 questions that are not about time, medium effort scores 76.0 and maximum effort 77.5 (the interval of the difference, −3.7 to +6.9, includes zero). On the 60 questions about time, medium effort scores 38.0 and maximum effort 53.5 (+15.4, 95% interval +7.9 to +23.0). Those 60 questions are also most of what maximum effort costs: $0.70 per question against $0.21 for the others.
And Scroll 2 at medium effort?
Scroll 2 itself, with the reader at medium effort, scores 60.11, with 32% of the marked evidence shown. Its first list holds 26% to 28% of that evidence in its two runs. At medium effort Zeta is +4.5 points above Scroll 2 (95% interval −1.1 to +10.1), close to the difference at maximum effort, with an interval that includes zero. Lowering the reader from maximum to medium effort costs both about the same: 5.4 points for Scroll 2 and 5.7 for Zeta.
Every run in numbers
| Model and reader effort | Score | Cost | Seconds | Memory tokens | Tool calls | Output tokens (of which thinking) | Evidence shown |
|---|---|---|---|---|---|---|---|
| Zeta, low | 64.64 | $0.029 | 10 | 1,092 | 0.69 | 403 (89) | 52.2% |
| Zeta, medium | 64.58 | $0.035 | 10 | 1,386 | 1.25 | 559 (189) | 53.0% |
| Zeta, high | 64.77 | $0.043 | 11 | 1,618 | 1.67 | 674 (257) | 54.8% |
| Zeta, xhigh | 63.40 | $0.062 | 16 | 2,269 | 2.74 | 1,116 (598) | 56.8% |
| Zeta, max, first | 70.39 | $0.353 | 103 | 5,556 | 7.12 | 10,519 (9,639) | 65.1% |
| Zeta, max, second | 70.21 | $0.341 | 102 | 5,332 | 6.87 | 10,181 (9,312) | 66.9% |
| Zeta, max, third | 72.52 | $0.340 | 101 | 5,031 | 6.51 | 10,268 (9,389) | 67.4% |
| Zeta, max, off | 69.25 | $0.339 | 101 | 4,835 | 6.98 | 10,279 (9,419) | 58.6% |
| Zeta, medium, off | 61.00 | $0.036 | 19 | 1,352 | 1.46 | 545 (175) | 48.4% |
| Scroll 2, medium | 60.11 | $0.041 | 10 | 1,908 | 1.25 | 567 (176) | 32.3% |
| Scroll 2, max | 65.48 | $0.332 | 96 | 5,794 | 4.99 | 9,712 (8,970) | 47.0% |
| Tablet 2, max | 52.95 | $0.132 | 42 | 2,679 | 0 | 4,658 (4,408) | 33.3% |
| Scroll 1.2, max | 53.09 | $0.083 | 26 | 1,872 | 0 | 2,739 (2,500) | 26.1% |
| Tablet 1, max | 44.65 | $0.041 | 16 | 387 | 0 | 1,617 (1,439) | 21.1% |
| Scroll 1, max | 38.23 | $0.032 | 12 | 337 | 0 | 1,179 (1,020) | 13.9% |
Cost, seconds, tokens and tool calls are per question. “Off” is a run with one part of Zeta switched off, and the third run was made after one more change to Zeta. The reader's cost is each question's usage record at Claude Opus 5.5's list price; the runs themselves were answered on a subscription.
03
One part switched off
What is that part worth?
We switched one part of Zeta off and ran again, in two conditions. With the reader at maximum effort, having it on is worth +1.1 points. With the reader at medium effort it is worth +3.6. Both intervals include zero, the second only narrowly.
| Condition | On | Off | Difference | 95% interval |
|---|---|---|---|---|
| Reader at maximum effort | 70.30 | 69.25 | +1.05 | −1.8 to +3.9 |
| Reader at medium effort | 64.58 | 61.00 | +3.58 | −0.4 to +7.5 |
The two conditions differ in how much the reader searches again. At maximum effort it calls the memory tools about seven times for a question, so what came up first decides less of the score. At medium effort it calls them about once.
That part is not the only difference between Zeta and Scroll 2. With it off and the reader at maximum effort, Zeta scores 69.25, +3.8 points above Scroll 2 (95% interval −1.1 to +8.7). With it off and the reader at medium effort, Zeta scores 61.00 and Scroll 2 60.11 (+0.9, 95% interval −4.6 to +6.4). Both intervals include zero. This remainder comes from the other differences in Zeta, which this note does not describe.
What changes when it is on?
What the reader is handed first changes. The first list has 14 lines on average, and with the part on, 5.4 of them on average are different lines. Over the 200 questions, 19 of the lines that left were lines of evidence, and 39 of the lines that came in were. The first list holds about 51% of the marked evidence with the part on and 44% with it off.
With the reader at maximum effort, the share of evidence it was shown over the whole question rose from 58.6% to between 65% and 67%, and the questions where it saw none fell from 31 to between 15 and 20. Instruction following is 91.2 with the part off and 98.8 with it on, in both runs (+7.5, 95% interval −0.3 to +15.3, which includes zero; twenty questions).
What did it cost, and how far was it in effect?
With the reader at maximum effort the total barely moves. Having the part on also costs about 600 more tokens of memory per question (4,835 to 5,444).
In the BEAM-10M runs the part was only partly in effect: for about 30% of what it applies to, and fully in two of the ten conversations. On the 40 questions of those two conversations, at maximum effort, the runs with it on scored 64.0 and the run with it off scored 67.9 (−3.8, 95% interval −8.6 to +1.0). On the other eight conversations the difference is +2.3 (−1.0 to +5.5). On BEAM-10M we have not measured it on and off while fully in effect.
04
The models since July
How do the earlier models score on the same store?
We also asked the four models before Scroll 2 the same 200 questions with the reader at maximum effort. They read the same stored conversations. They have no memory tools, so each was asked in its own configuration and the reader answered from what came back, without searching again. Their answer prompt lacks one sentence that the Scroll 2 and Zeta runs carry, which tells the reader to combine the memories it has; its effect on these models was not measured. Tablet 2 scores 52.95 with 2,679 tokens of memory, Scroll 1.2 53.09 with 1,872, Tablet 1 44.65 with 387, and Scroll 1 38.23 with 337. The share of the marked evidence in front of the reader is 33%, 26%, 21% and 14%.
These numbers are not differences under the same conditions as the rest of this note. Scroll 2 at maximum effort is +12.5 points above Tablet 2 (95% interval +7.6 to +17.4), but one reader could search again and the other could not. Scroll 1 and Scroll 1.2 were measured as engines alone, without the hosted service they were sold with, so their numbers may be below what the products would have scored.
What have we asked since July?
The first report, in July, measured on LongMemEval-S: 500 questions, each carrying about 140,000 tokens of earlier conversation. The right memory was found 99.6% of the time, and the answers scored 85.2% for Tablet 1 and 90.7% for Scroll 1. The 9 to 14 points between found and answered were what that report was about. Scroll 1.2 scored 92.3% on the same benchmark.
Tablet 2, in August, widened the engine to languages and images, and was the first we measured on BEAM-1M. Much of that report was about how little one score says on its own: one setting that reports rarely state moved BEAM-1M by 8.9 points. Scroll 2, in November, scored 4.6 points above Tablet 2 per question on BEAM-1M. That report measured apart what the engine handed over, what the reader used, and how the benchmark scores, and it named weak points: temporal reasoning, event ordering, and saying that the record does not hold the answer.
| Published | Model | Benchmark and score | Against the model before it, same conditions |
|---|---|---|---|
| 7 and 15 July | Tablet 1, Scroll 1 | LongMemEval-S 85.2, 90.7 | The right memory was found 99.6% of the time |
| 19 July | Scroll 1.2 | LongMemEval-S 92.3 | +1.6 over Scroll 1. Five runs each; the readers differ, to Scroll 1.2's disadvantage |
| 24 August | Tablet 2 | LongMemEval-S 95.7, BEAM-1M 67.5 | Read by the same model, Scroll 1.2 scores 92.3 and Tablet 2 93.7. Separate campaigns, so not reported as an established difference |
| 5 November | Scroll 2 | BEAM-1M 72.1, and 73.2 with memory tools | +4.6 over Tablet 2 (95% interval +3.4 to +5.9); with memory tools +5.7 (+4.1 to +7.2) |
| This note | Zeta, experimental | BEAM-10M 70.3 with memory tools | +4.8 over Scroll 2 (95% interval +0.7 to +8.9) |
What about BEAM-1M?
We ran Zeta once on BEAM-1M, 700 questions, as it stood in the third run, with the same reader at maximum effort and the same judge. In this run the part that can be switched off was fully in effect. Zeta scores 74.62 by BEAM's scoring and 79.31 with event ordering graded by rubric, with 3,974 tokens of memory per question and 5.75 tool calls.
The Scroll 2 report gives 73.2 for Scroll 2 with memory tools on this benchmark, read by Claude Opus 5. The readers differ, so the 1.4 points between the two are not a difference under the same conditions. We have no run of Scroll 2 on BEAM-1M with this reader and memory tools, so this run does not tell whether the difference seen on BEAM-10M holds at this size.
05
Reading the score
How far does the same run move?
We ran the same model with the same settings and the same reader twice. The totals are 70.39 and 70.21, 0.2 points apart. By question, 68 of the 200 scored differently, 39 higher in the second run and 29 lower. One ability is 20 questions, so it moves further: preference following was 90.8 and 78.8, temporal reasoning 62.5 and 68.8, abstention 77.5 and 85.0. So this note reads a difference in one ability only where both runs point the same way.
What does the score not count?
The Tablet 2 report said the event-ordering score measures something other than ordering. This time we counted it. The benchmark cuts an answer into lines, asks of each line whether it is the same thing as a rubric item, pairs them, and then looks at the order. Graded item by item, 70.6% of the rubric items are present in Zeta's answers. Line matching paired 34.1% of the rubric items. Among the paired items the order scores 89.7 by the benchmark's measure. For Scroll 2 the three figures are 44.8%, 17.8% and 65.2. A rubric item is a short name; a line of the answer is a sentence with a date and what happened.
In temporal reasoning, in Zeta's first run, seven questions had the number of days right and both dates written in the answer, and lost the rubric item that asks for the range from one date to the other. That is 1.4 points of the total. We did not shape the answers to the rubric. The instruction given to the reader is the one used to measure Scroll 2.
06
Limits
Where is it weak?
A weak point the Scroll 2 report named is still there. On saying that the record holds no answer, Scroll 2 scores 95.0 and Zeta 77.5 and 85.0. One reading is that when more is found, the reader builds an answer from it even where the record holds none; we did not measure this.
Zeta scores below Scroll 2 on multi-session reasoning: 39.6 and 37.2 in its two runs, against 45.0.
Some questions are heavy. For an event-ordering question the reader receives 11,912 tokens on average and calls the tools 13 times. At maximum effort the reader costs $0.35 and $0.34 per question in Zeta's two runs, against $0.33 with Scroll 2.
Zeta's configuration was chosen on BEAM-10M, and BEAM-1M was measured afterwards with the same configuration. BEAM is in English, and scores in other languages were not measured. The new measurements in this note use one family of benchmarks, one reader and one judge.
Zeta is an experimental model. It is not a released product.
Who wrote this
This is our own product and this note was written by the person who builds it. Every number in it is self-reported.
Notes
- Intervals
- For the new measurements, every interval is the mean per-question difference plus or minus 1.96 standard errors, paired per question. The intervals quoted from earlier reports are those reports’ own.
- Earlier numbers
- The figures for Tablet 1, Scroll 1, Scroll 1.2, Tablet 2 and Scroll 2 on LongMemEval-S and BEAM-1M are those of the five earlier reports, unchanged.
- Models
- Scroll 2 and the earlier models are measured as they stood on 8 November 2026. Zeta starts from our engine as it stood on 10 November 2026. Scroll 2 is the released model. Every model reads the same stored conversations. Scroll 2 and Zeta are used with their memory tools; the models before Scroll 2 have none.
- Runs
- Zeta was run twice with the same settings, and every other condition once. The third run was made in two parts, the 60 questions about time and then the other 140.
- BEAM
- BEAM (Beyond a Million Tokens, ICLR 2026) is released under CC BY-SA 4.0.