Gödel Agent — Recursive runtime self-modification¶
Self-edit. The artifact owns both its solve prompt and the self-improvement prompt that rewrites it, so the search rewrites its own search. Runs through the shared
MethodPolicyrunner, behind an AST gate. Example:examples/godel_agent/godel_agent_self_modify.py.
| Paper | Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement — Yin et al., 2024 (arXiv:2410.04444) |
| Upstream code | Arvid-pku/Godel_Agent@bbb50879 |
| Example | examples/godel_agent/godel_agent_self_modify.py |
| Domain | GSM-Hard (reasoning-machines/gsm-hard), 64/64/64 shuffled splits; two AST-gated policy functions |
| Layer | L1 (blast_radius=0.6, set by the shared runner) |
| Fidelity | self_edit_analogue — what the classes mean |
This port is measured in the runtime matrix: the mechanism is preserved and measured under AgentDescent's runtimes; it is not a paper-benchmark reproduction.
The mechanism¶
Gödel Agent monkey-patches its own logic in runtime memory: it reads its own source (self-awareness via inspecting Python runtime state) and rewrites it mid-execution. There is no acceptance gate upstream — edits are kept unless they crash, recovery is error-triggered only, and the paper reports 92% of trials dipping before improving. Steering comes from optional self-evaluation.
Where each piece lives¶
| Upstream mechanism | Where it lives here |
|---|---|
| Artifact-owned recursion | the artifact holds both solve_prompt and self_improvement_prompt; each proposal is generated by the artifact's own current self-improvement prompt applied to its own source |
| Gatelessness | --gateless installs an accept-any-compiling AcceptancePolicy; the default keeps the framework gate and labels it a substitution |
| Monkey-patching | AST-gated whole-source replacement |
Boundaries¶
- The held-out gate (default) is a deliberate substitution, not a preserved mechanism.
- AST-gated replacement stands in for monkey-patching a full scaffold.
Measured results — GSM-Hard¶
Three seeds, async_pipeline, 80 rollouts each, 8 workers, --staleness full,
reflective merge on (the policy default, and
validated against the same AST gate the
proposals pass), with the framework gate (the default, and a declared
substitution -- see below), deepseek-v4-flash at temperature 0.7, 64/64/64
shuffled splits. Recorded in
bench/results/godel-agent-self-modify.json.
| seed | test quality | validation | accepted | invalid | calls |
|---|---|---|---|---|---|
| 0 | 0.672 → 0.750 | 0.547 → 0.656 | 2/80 | 14 | 941 |
| 1 | 0.641 → 0.609 | 0.578 → 0.656 | 4/80 | 3 | 1007 |
| 2 | 0.562 → 0.703 | 0.625 → 0.781 | 3/80 | 13 | 1006 |
Mean gain +0.062 on test and +0.114 on validation. Seed 1 regresses by 0.032, which is inside the ±0.06 across-draw sd at n=64 and is the shape a gate that judges on a different draw is expected to produce sometimes. See the caveat on PromptBreeder: one run per seed does not pin a number here either.
The invalid counts swing from 3 to 14 because the editable surface is two AST-gated functions where SICA has one, so a proposal that names only one of them, or writes a body the node whitelist refuses, is counted rather than merged. The same split fix described on SICA applies to this row: these numbers are from the shuffled-split run.
Run it¶
python -m examples.godel_agent.godel_agent_self_modify --dry-run
# one seed of the three above
python -m examples.godel_agent.godel_agent_self_modify --yes --seed 0 \
--provider openai --model deepseek-v4-flash \
--async --async-ratio 1 --workers 8 --budget-rollouts 80 --staleness full \
--temperature 0.7 --max-seconds 3600
# the control: upstream's own gatelessness, any compiling edit kept
python -m examples.godel_agent.godel_agent_self_modify --gateless --dry-run
--async-ratio 1 is what this row ran at: the flag was dropped before it
reached the runtime, so the run took the runner's default whatever the
command line said. It is passed explicitly here because the value matters and
a default can move.
Flags: the MethodPolicy command line.
Offline tests: tests/test_godel_upstream.py,
tests/test_candidate_methods.py.