GEPA: How language learns to improve itself

§ 2The central insight

Information per rollout, not number of rollouts, is the lever that matters.

Imagine running your prompt once and getting back a score of 0.3. That number tells you the run failed. It does not tell you why. Was the reasoning wrong? Did the tool call return the wrong format? Did the model refuse halfway through? Did the final answer get the entities right but the dates wrong?

Now imagine running it once and getting back a score of 0.3 plus a paragraph: "The model retrieved the correct two passages but failed to synthesize them — it ignored the second passage entirely and answered based only on the first. The phrasing 'briefly answer' may be causing premature closure." That second form of feedback is what GEPA's reflection step consumes. It is dramatically more actionable than a number.

This reframing of the optimization problem — treat each rollout as a diagnostic text, not a reward signal — is what makes GEPA sample-efficient. The authors find that GEPA reaches better performance than RL with 300–1200 rollouts where RL needs 24,000+.

§ 3Three pillars

The full system rests on three ideas working together. Each is simple in isolation; their combination is what makes the algorithm distinctive.

The next three sections unpack each pillar with an interactive demo. We start with Pareto selection, because it is the idea most often misunderstood.

§ 4Why Pareto selection?

The naive strategy in an evolutionary search is to always mutate the currently-best candidate. Pick the prompt with the highest average score, mutate it, see if you can do better. Repeat.

This strategy fails in a specific and instructive way. Suppose prompt P1 averages 0.62, prompt P2 averages 0.58, and prompt P3 averages 0.55. P1 wins. We mutate it. But what if P2 and P3 each contain a useful strategy that P1 lacks — a strategy that happens to be averaged out across the dataset? By always mutating P1, we discard those strategies. The search collapses.

GEPA's fix is borrowed from multi-objective optimization. Treat each training example as its own objective. A prompt is Pareto-optimal — said to lie on the Pareto frontier — if no other prompt beats it on every example. Equivalently: a prompt is on the frontier if it is the best (or tied for best) on at least one example. The frontier is the set of complementary winners.

The table below shows three candidate prompts evaluated on four tasks. Click "Show frontier" to highlight which candidates are Pareto-optimal — and notice that the candidate with the highest average is not the only one we want to keep.

The geometric picture

If we project the same data onto two of the tasks at a time, the frontier becomes visible as a literal frontier — the upper-right boundary of the cloud of points. A candidate that sits "inside" the cloud is dominated: some other candidate beats it on every dimension shown. A candidate on the boundary is non-dominated. Use the controls to switch between task pairs and to add new candidates one at a time.

§ 5Reflective prompt mutation

Once a parent prompt has been selected from the Pareto frontier, GEPA needs to mutate it into a child. Classical genetic algorithms would apply random perturbations: swap a token, insert a phrase, splice in part of another prompt. GEPA does something far more pointed.

It runs the parent prompt on a small minibatch of training examples (typically just three). For each example it captures:

• the input,
• the model's full output trace (including intermediate reasoning and tool calls),
• a textual feedback string from the metric function — not just a score, but a description of what was right or wrong.

All of this is packaged into a meta-prompt and sent to a reflection LLM, which is asked to write a better instruction for the system. Here is the actual template used in the paper:

// The reflection meta-prompt (paraphrased from the paper)

I provided an assistant with the following instructions to perform a task for me:
```
<current instruction>
```

The following are examples of different task inputs provided to the assistant
along with the assistant's response for each of them, and some feedback on
how the assistant's response could be better:
```
<Inputs, Outputs and Feedback for minibatch of examples>
```

Your task is to write a new instruction for the assistant.

Read the inputs carefully and identify the input format and infer detailed
task description about the task I wish to solve with the assistant.

Read all the assistant responses and the corresponding feedback. Identify
all niche and domain-specific factual information about the task and include
it in the instruction, as a lot of it may not be available to the assistant
in the future. The assistant may have utilized a generalizable strategy to
solve the task, if so, include that in the instruction as well.

Provide the new instructions within ``` blocks.

Notice what this meta-prompt is doing. It is not asking the reflection model to "improve the prompt." It is asking it to extract latent task knowledge from the trajectories — facts and strategies that worked, and to compile them into the new instruction. Each successful mutation makes the prompt slightly more domain-specialized.

Below is a worked example. Step through it to see how a generic instruction becomes a sharper one after one round of reflection.

§ 6The full algorithm, one step at a time

Now that the two key pieces — the frontier and the reflective mutation — are in hand, we can assemble the full optimization loop. Step through the eight stages below to see how a single iteration of GEPA unfolds.

§ 7Watching prompts evolve

Each surviving candidate is born from a parent (and occasionally two parents in a merge). Over the course of optimization, this produces a literal genealogy tree. The figure below is a stylized version: the root is the seed prompt, each node is a candidate, and the radius encodes its aggregate validation score. Click "Run iteration" to grow the tree.

§ 8Results

Across four benchmarks reported in the paper — HotpotQA (multi-hop QA), IFBench (instruction following), HoVer (fact verification), and PUPA (privacy-preserving delegation) — GEPA outperforms reinforcement learning (GRPO) by up to 19 percentage points while consuming a small fraction of the rollout budget. It also beats MIPROv2, the previous state-of-the-art prompt optimizer in the DSPy ecosystem.

§ 9Why it works

Four properties of GEPA explain its sample efficiency. None of them is novel in isolation — together they form a distinctive method.

Where GEPA fits

GEPA is implemented as a first-class optimizer in DSPy (dspy.GEPA) and as a standalone Python package at gepa-ai/gepa. It also appears in DeepEval's prompt-optimization toolkit. The library is general: any text component of any system can be evolved against any measurable metric — prompts, configuration files, even snippets of code or vector-graphic specifications.

"If you can measure it, you can optimize it."
— gepa-ai/gepa, project README

The work has been positively noted by industry figures — Tobi Lütke of Shopify singled out GEPA and DSPy as "severely underhyped" — and the paper was accepted at ICLR 2026 as an oral presentation.

Where it might fall short

GEPA's strength is also its dependency: it needs a good reflection LLM and a well-designed feedback function. If the reflection model is weak, the mutations will be poor. If the feedback function returns only scalars, GEPA degrades toward something closer to a vanilla evolutionary search. The paper recommends a frontier model (e.g. GPT-5 at high temperature) as the reflection LM — meaning the cost saved on rollouts is partly spent on a more capable reflection model. For tasks where verifiable rewards are easy but rich feedback is hard to construct, classical RL may still be competitive.