Absolute Zero — Zero-data self-play (single model)¶
Self-play policy evolution. One model proposes and solves its own tasks, graded by a local verifier and rewarded for learnability. Runs through the shared
MethodPolicyrunner. Example:examples/absolute_zero/absolute_zero_selfplay.py.
| Paper | Absolute Zero: Reinforced Self-play Reasoning with Zero Data — Zhao et al., 2025 (arXiv:2505.03335) |
| Upstream code | LeapLabTHU/Absolute-Zero-Reasoner@484afa48 |
| Example | examples/absolute_zero/absolute_zero_selfplay.py |
| Domain | self-generated cart arithmetic — 16 self-play slots + 16/16 frozen evaluation carts (deduction + abduction) |
| Layer | L1 (blast_radius=0.6, set by the shared runner) |
| Fidelity | inference_analogue — what the classes mean |
This port is measured in the runtime matrix: the mechanism is preserved and measured under AgentDescent's runtimes; it is not a paper-benchmark reproduction.
The mechanism¶
One model plays both proposer and solver over Python-verifiable task
triplets in three modes (deduction, abduction, induction), with a code executor
as the grounded verifier. The proposer's reward is learnability: 1−r̄ when
the solver's success rate r̄ is positive, zero at both extremes — monotone in
difficulty, not peaked at 50%. Both roles update the same weights via TRR++
(task-relative baselines per task-type × role).
Where each piece lives¶
| Upstream mechanism | Where it lives here |
|---|---|
| Proposer/solver self-play | training rollouts generate and solve their own carts, verified by the local renderer |
| Grounded verifier | the deterministic cart renderer; reward uses the domain's tolerant integer parse |
Learnability reward 1−r̄ |
surfaced verbatim in the proposer update prompt (deliberately not mapped to DifficultyWeighted, whose 4p(1−p) peaks at 0.5) |
| Frozen evaluation | held-out/test carts generated from the seed at build time — the evolved memory cannot shape its own test set |
Boundaries¶
- Verbal policy memory replaces TRR++ weight updates.
- Deduction and abduction stand in for the paper's three task types; induction is omitted.
Measured results — self-play carts¶
Three seeds, async_pipeline, 80 rollouts each, 8 workers, --staleness full,
reflective merge on (this method's own declaration), two solver samples per
proposed item, deepseek-v4-flash at temperature 0.7. Recorded in
bench/results/absolute-zero-selfplay.json.
| seed | test quality | validation | accepted | calls |
|---|---|---|---|---|
| 0 | 0.125 → 0.375 | 0.125 → 0.438 | 4/80 | 575 |
| 1 | 0.000 → 0.438 | 0.062 → 0.438 | 3/80 | 657 |
| 2 | 0.000 → 0.250 | 0.000 → 0.375 | 2/80 | 641 |
All three seeds moved; mean gain +0.313, mean final 0.354.
The baseline is not 0.000 here, unlike the money-domain rows. The evaluation carts are generated by a trusted renderer rather than gated on discovering one output convention, so a fresh solver already gets some of them — and the ceiling is not 1.000 either. Read the gain, not the final value, against the other pages.
See the caveat on PromptBreeder: one run per seed does not pin a number here either.
Learnability is per item, which is why the solver is sampled twice
Upstream's accuracies[uid] is the solve rate of that problem over its
rollout group, so 1 − r̄ is that item's learnability, not the run's. One
solver sample makes the rate 0 or 1, and (1 − r̄) if r̄ > 0 else 0 is zero
at both — so two samples is the smallest number that produces a middle.
The formula stays monotone in the solve rate and not peaked at 0.5, which is why no difficulty-weighted sampler is attached here where R-Zero and Agent0 have one.
Run it¶
python -m examples.absolute_zero.absolute_zero_selfplay --dry-run
# one seed of the three above
python -m examples.absolute_zero.absolute_zero_selfplay --yes --seed 0 \
--provider openai --model deepseek-v4-flash \
--async --async-ratio 1 --workers 8 --budget-rollouts 80 --staleness full \
--temperature 0.7 --max-seconds 3600
--async-ratio 1 is what this row ran at: the flag was dropped before it
reached the runtime, so the run took the runner's default whatever the
command line said. It is passed explicitly here because the value matters and
a default can move.
Flags: the MethodPolicy command line.
Offline tests: tests/test_selfplay_upstream.py,
tests/test_candidate_methods.py.