GEPA — Reflective Prompt Evolution

Skill / prompt self-evolution. Evolve an instruction prompt with a genetic, reflective loop whose parent selection is a per-instance Pareto frontier. Runs through evolve() with a custom aggregator_factory. Example: examples/gepa/gepa_prompt_evolution.py.

Paper	GEPA: Reflective Prompt Evolution Can Outperform RL — Agrawal et al., 2025 (arXiv:2507.19457)
Repo	gepa-ai/gepa (also dspy.GEPA)
Dataset	HotpotQA (multi-hop QA, distractor), exact match
Layer	L2 prompt (blast_radius=0.2)

The algorithm

Two distinctive mechanisms, both preserved:

1. Reflective mutation (Algorithm 1 UpdatePrompt). On a failure the LLM reflects on the execution trace and the natural-language feedback (μ_f: predicted vs. gold), then writes a new instruction. This is the propose step.
2. Pareto-based candidate selection (Algorithm 2) — the reason GEPA beats greedy hill-climbing. Instead of always mutating the single best-average candidate, it keeps a pool scored on every D_pareto instance and samples the next parent from the per-instance Pareto frontier, weighted by how many instances a candidate uniquely wins — keeping complementary specialists alive.

pareto_frontier / pareto_select implement Algorithm 2 faithfully (per-instance best → union of winners → dominance pruning → frequency-weighted sampling) and are unit-tested.

How it plugs into evolve()

The greedy evolve() loop always mutates the dev head; GEPA needs to mutate the Pareto-selected parent. A custom aggregator_factory (ParetoAggregator) supplies that: it scores each candidate on the held-out D_pareto, runs Algorithm 2, and commits the sampled parent as the dev head, so the next round mutates it. This is the sanctioned "swap the whole optimizer" hook.

factory = pareto_aggregator_factory(artifact_id="gepa_prompt", seed=0)
evolve(tasks, reward, agent=gepa_agent(completion),
       strategy=InstructionSlot(), aggregator_factory=factory, blast_radius=0.2)
best = factory.holder["agg"].best_state["instruction"]   # GEPA returns best-average

Fidelity notes

GEPA optimises a multi-module compound system with a rollout budget; here the system is a single instruction module and the minibatch is the per-round worker sample (raise --workers). The Pareto set is the held-out split.

Plug-ins implemented

In examples/gepa/gepa_prompt_evolution.py:

Plug-in	evolve() slot	What it does
InstructionSlot	strategy=	a single evolvable instruction module; each proposal replaces it (content-addressed)
ParetoAggregator / pareto_aggregator_factory	aggregator_factory=	GEPA's per-instance Pareto candidate selection; commits the sampled Pareto parent as the dev head
pareto_frontier / pareto_select	(pure, unit-tested)	Algorithm 2: per-instance best → union of winners → dominance pruning → frequency-weighted sampling
gepa_agent()	agent=	Generator + reflective-mutation actor (rewrites the instruction from trace + NL feedback)

Measured — HotpotQA with DeepSeek

--rounds 5 --fetch 40 --provider openai --model deepseek-v4-flash:

	exact match
seed instruction, on the Pareto set	0.500
best candidate found	0.600
test set (held out, never seen by the optimizer)	0.700

4 candidates explored, 80 model calls, ~10 min wall-clock. The instruction it found:

"Read the context carefully and connect information across multiple paragraphs to identify who matches all the clues in the question. Then give only the final answer as a short phrase, without explanation."

Both halves of that are real HotpotQA failures: multi-hop evidence, and a model that answers a short-span question with a paragraph. This is the one shipped port whose benchmark still has headroom for deepseek-v4-flash — see Measured results for why the others do not.

Run it

python -m examples.gepa.gepa_prompt_evolution --dry-run
python -m examples.gepa.gepa_prompt_evolution --model claude-haiku-4-5

Offline tests: tests/test_gepa_example.py (incl. the Algorithm-2 selection).