Topic Prompt optimization Framework DSPy · Stanford NLP Paper Opsahl-Ong et al., 2024

An interactive explainer

MIPROv2

A practical algorithm that automatically writes better prompts for multi-stage language model programs — tuning both the natural-language instructions and the few-shot examples for every prompt in the pipeline, jointly, by Bayesian search.

Why it exists The three phases Bootstrap Propose Search & TPE Search-space size Vs. siblings When to use

§ 1.0

What it’s solving

Why hand-tuning prompts in a pipeline falls apart

A real LLM application is almost never a single prompt. It’s a pipeline — several language-model calls strung together, each doing one piece of the job. A retrieval-augmented question answerer might first generate a search query, then retrieve documents, then read them and answer. Each of those steps is a separate prompt that needs its own instructions and its own examples.

For each of those prompts, you have two knobs you can turn:

The instruction — the natural-language description of what this prompt should do.
The few-shot examples — sample input/output pairs that demonstrate the behaviour.

Tuning these by hand is painful for one prompt, and quickly impossible for several. The instruction in step 2 depends on what step 1 produces, and good examples for step 3 depend on what step 2 returns. The knobs are coupled. So the question is: can we automate this? That’s exactly what MIPROv2 does.

Click any module below to see what MIPROv2 will optimize for it.

Figure 1 · A two-stage RAG program

Module 1

GenerateQuery

Instruction

Few-shot demos

Module 2

Retrieve

(no LM here)

Module 3

Answer

Instruction

Few-shot demos

Click a module above

Each LM module has two tunable knobs. MIPROv2 will jointly search over all of them at once.

A pipeline with two LM-backed modules (GenerateQuery and Answer) has four knobs to tune. MIPROv2 optimises them together rather than one at a time, because they interact: a better Answer instruction is only as good as the queries that feed it.

§ 2.0

The recipe, in three phases

Bootstrap · Propose · Search

MIPROv2 runs in three sequential phases. The first two generate candidates — pools of plausible few-shot demos and plausible instructions for each module. The third picks the best combination of those candidates by treating the choice as a Bayesian optimization problem.

§ 3.0

Phase 1 · Bootstrap demonstrations

Filter program traces by success

This phase is borrowed almost verbatim from an older optimizer called BootstrapFewShot. The idea is dead simple, and very pretty:

Run the current (un-optimized) program on a batch of training examples.
For each run, you get a complete trace — the input/output pair for every module in the pipeline.
Score the final output with your metric. If the program got the right answer, the entire trace is a success.
For each module, harvest the (input, output) pair from successful traces. Those become candidate few-shot examples for that module.

The clever bit: you don’t need labels for the intermediate modules. The end-to-end metric tells you the final answer was right, and you assume the intermediate steps that led to it must have been reasonable. Then you repeat this many times with different random samples, producing several distinct sets of demonstrations per module — a pool to choose from later.

Why it works Bootstrapping creates examples in the exact format your program produces, not what a human guesses it produces. The demos already speak the right schema.

Live · Bootstrapping traces

Kept: 0 / 0

Each row simulates running the program on one training example. Green = metric passed, the trace is kept as a demo candidate. Red = failed, discarded.

§ 4.0

Phase 2 · Propose instruction candidates

A “grounded proposer” LM writes new instructions for each module

If bootstrapping built a pool of demo candidates, this phase builds a pool of instruction candidates. The trick is doing it well.

A naive optimizer would just ask an LM “write me 10 instructions for this prompt”. MIPROv2’s key innovation here — and the reason it’s “v2” — is that the proposer is grounded. It doesn’t generate in a vacuum. Before writing anything, the proposer is shown:

The program’s code — so it understands what each module does in context.
A summary of the dataset — a few examples and a synthesised description of the task.
The bootstrapped demos for this module — so the instruction it writes is consistent with the examples it’ll be paired with.
A randomly chosen “tip” — like “be concise”, “be creative”, “think step by step”. This nudges diversity across candidates.
A temperature setting — sampled randomly, also for diversity.

The output is a set of N candidate instructions per module — each one different, each one informed by what the module actually needs to do.

Subtle point The proposer uses a separate LM (often a stronger one) than the model that will run the program. You’re using a capable model as a prompt engineer for a smaller model.

The grounded proposer

ProgramRAG pipeline source code

Data summary“Open-domain QA over Wikipedia…”

Demos3 bootstrapped examples from §3

Tip“be concise”

Temp0.7

↓

Proposer LM

↓

Candidates

For module Answer

§ 5.0

Phase 3 · Bayesian search over combinations

Tree-structured Parzen Estimator (TPE) via Optuna

By now you have, for each module, a pool of candidate instructions (say 10 of them) and a pool of candidate demo sets (also 10). The question is: which combination is best?

If you have M modules, each with I instruction options and D demo options, the number of complete configurations is:

(I × D)^M

With 2 modules, 10 instructions each, and 10 demo sets each, that’s 10,000 configurations — far too many to evaluate exhaustively. We need to be smart. We need Bayesian optimization.

What “Bayesian optimization” actually means here

Strip away the jargon. Bayesian optimization is just a loop:

Pick a configuration to try.
Evaluate it — run the program on a small mini-batch and score it with your metric.
Use everything you’ve seen so far to make a better guess about which configuration to try next.

The thing that makes it Bayesian is step 3. You’re building, on the fly, a probabilistic model of “configurations that work well” vs. “configurations that don’t”, and using it to bias your next pick. MIPROv2 uses a particular technique called TPE for this.

TPE, demystified

TPE stands for Tree-structured Parzen Estimator. The name is intimidating; the idea isn’t. Here’s the whole algorithm in one sentence:

Split the trials you’ve seen into “good” and “bad”, learn what the good ones look like, and sample your next trial to look like a good one.

More concretely:

After every trial, sort your scores. Mark the top γ fraction (often the top 25%) as “good”, the rest as “bad”.
For each search dimension — e.g. “which of the 10 instructions does the Answer module use?” — build two distributions:
- ℓ(x): a histogram of which choice the good trials made.
- g(x): a histogram of which choice the bad trials made.
For the next trial, sample candidate configurations and pick the one that maximises ℓ(x) / g(x) — the choice that’s most overrepresented among winners relative to losers.

That ratio is just “promising-ness”. A choice that appears often in good trials and rarely in bad ones has a high score. As more trials accumulate, the histograms get sharper, and TPE concentrates its picks in the promising region of the search space.

The “tree-structured” part Configurations are hierarchical: choosing one module’s instruction is independent of choosing another’s. TPE models them with a tree of independent histograms rather than one giant joint distribution, which is why it handles MIPROv2’s combinatorial space gracefully.

Why Parzen? For continuous parameters, TPE would use Parzen estimators (kernel density estimates) — bumps centred on each trial. For discrete parameters like MIPROv2’s, those bumps collapse to weighted counts of categorical choices.

Watch it converge

Each dot below is one trial. The x-axis is “configuration” (an abstract index over the search space, 0 to 1); the y-axis is the metric score. Click Run trial to draw a new configuration. Watch where the dots start landing as more trials accumulate — TPE pulls them toward the region where the high scorers live.

Figure 2 · TPE search, animated

Trial

Best score

—

Strategy

Random warm-up

Trial Top 25% (good) Latest TPE preference density

The first ten trials are uniform random (Optuna’s default warm-up). After that, TPE kicks in: it fits its ℓ/g distributions over the trials so far and biases new picks toward the part of the x-axis where the top scorers cluster. The shaded ridge shows where TPE currently thinks “good” configurations live.

One more wrinkle: mini-batching

Evaluating a configuration on the full validation set is expensive — each evaluation runs the entire pipeline once per example. MIPROv2’s default is to evaluate each trial on a small mini-batch (say 25 examples) instead, which is noisier but ~10× cheaper. Periodically, the best-performing configuration so far is re-evaluated on the full validation set to confirm it’s actually best.

This is the standard noisy-Bayesian-optimization trick: spend cheap, noisy evaluations to explore, spend expensive, accurate evaluations only on the leading contenders.

§ 6.0

How big is the search space, really?

A live calculator for combinatorial explosion

Drag the sliders to see how the configuration space scales. It’s why we need a smart optimizer — even a modest pipeline produces a space too large to grid-search.

Modules in the pipeline

Instruction candidates per module

Demo sets per module

Total configurations

10,000

(10 × 10)²

A typical MIPROv2 run evaluates around 50–100 of these, not all of them.

§ 7.0

How it compares to its siblings

The DSPy optimizer family

MIPROv2 didn’t come from nowhere. It’s the joint successor to two simpler optimizers in DSPy that each tackled half the problem. Understanding what they did well clarifies what MIPROv2 adds on top.

BootstrapFewShot

Demos only

✓Bootstraps successful traces into demos

×Doesn’t touch instructions

×No joint search across modules

✓Cheap, simple, fast

COPRO

Instructions only

✓Iteratively rewrites instructions

×Doesn’t touch demos

×Greedy, no Bayesian search

×Not grounded in program/data

The synthesis

MIPROv2

Joint — both knobs

✓Bootstraps demos (from BFS)

✓Proposes instructions (grounded)

✓Joint Bayesian search over both

✓Mini-batched for efficiency

There’s also a newer entrant, GEPA, that takes a different angle — using an LM to reflect on the program’s execution traces and propose targeted edits rather than running a Bayesian search over a fixed candidate pool. The two are complementary; MIPROv2 is the default workhorse when you just want “make my prompts better” without bespoke feedback signals.

§ 8.0

Practical notes

Using it in DSPy

In DSPy the API is one import, one constructor, one .compile(). The optimizer ships with three presets — light, medium, heavy — that tune the number of candidates and trials so you don’t have to.

When to reach for it

You have a DSPy program with one or more modules and a working metric.
You have a training set of at least a few dozen examples (more is better; ~150–500 is a typical sweet spot).
You can afford ~50–200 program runs for the search.

When not to

Your metric is genuinely noisy or hard to define — the optimizer is only as good as the signal you feed it.
You need very few examples (<20) — the bootstrap step won’t find enough successful traces.
You need detailed, targeted prompt edits with rich feedback signals — consider GEPA instead.

Minimal usage

import dspy
from dspy.teleprompt import MIPROv2

# 1. Define a program and a metric
program = dspy.ChainOfThought("question -> answer")
metric = my_eval_function

# 2. Instantiate the optimizer
optimizer = MIPROv2(
    metric=metric,
    auto="medium",   # light | medium | heavy
)

# 3. Compile against your trainset
optimized = optimizer.compile(
    program,
    trainset=my_trainset,
)

optimized.save("optimized.json")

Knobs worth knowing num_candidates — how many instructions/demo sets per module. num_trials — how many configurations TPE evaluates. minibatch_size — how many examples per trial. The auto presets set sensible defaults; touch the individual knobs if you know what you’re doing.

§ 9.0

The whole thing, in one paragraph

For when you need to explain it on a whiteboard

MIPROv2 optimises a multi-stage LM program by first running the un-optimized version on training data and keeping the traces that scored well — those become candidate few-shot demos. Then it asks a separate proposer LM, grounded in the program’s code and a summary of the data, to write multiple candidate instructions for each module. Finally, it treats every (instruction-choice, demo-set-choice) per module as a categorical hyperparameter and runs a Bayesian search — Optuna’s TPE — over the combinations, evaluating each on a mini-batch and concentrating its trials in the region of the search space where the high-scoring configurations live.

Bootstrap the demos. Propose the instructions. Search the combinations.