name

observable-gepa-migration

description

Migrate DSPy GEPA usage in DSPy from the original to gepa-observable. This makes it possible for teams to clearly review each iteration and the lineage to understand how their prompt is evolving. The repository offers a web dashboard for monitoring, but requires a custom GEPA fork that provides custom observers and LM call logging. Use when developers want to add observability to GEPA optimization.

Migrate DSPy GEPA to gepa-observable

Migrate existing DSPy GEPA code to gepa-observable for integrated observability: real-time dashboard, custom observer callbacks, and LM call capture working together.

Quick Migration

Step 1: Install gepa-observable

pip install dspy-gepa-logger

Step 2: Change Import

# Before
from dspy.teleprompt import GEPA

# After
from gepa_observable import GEPA

Step 3: Add Observability (Full System)

from gepa_observable import GEPA

optimizer = GEPA(
    metric=my_metric,
    auto="medium",
    # Observability system:
    server_url="http://your-server:3000",  # Dashboard integration
    project_name="My Project",
    capture_lm_calls=True,                  # LM call logging
    capture_stdout=True,                    # Console capture
    verbose=True,                           # LoggingObserver
    observers=[MyCustomObserver()],         # Custom callbacks
)

Step 4: Everything Else Stays the Same

Metric functions (5-arg signature already required by DSPy GEPA)
Program definitions (ChainOfThought, ReAct, etc.)
Training/validation data format
Budget params (auto, max_full_evals, max_metric_calls)
compile() call: optimizer.compile(student=program, trainset=train, valset=val)

The Observability System

gepa-observable provides an integrated observability system where all components work together:

Component	Purpose	Enabled By
ServerObserver	Sends events to web dashboard	`server_url` param
LoggingObserver	Console output with summaries	`verbose=True`
LM Call Logger	Captures all LM invocations	`capture_lm_calls=True`
Custom Observers	Your callbacks for any event	`observers=[...]`

All observers receive the same 8 lifecycle events:

SeedValidationEvent - Initial validation scores
IterationStartEvent - Each optimization iteration
MiniBatchEvalEvent - Minibatch evaluations
ReflectionEvent - Proposed prompt changes
AcceptanceDecisionEvent - Accept/reject decisions
ValsetEvalEvent - Full validation evaluations
MergeEvent - Candidate merge operations
OptimizationCompleteEvent - Final results

Custom Observer Example

class MyObserver:
    def on_seed_validation(self, event):
        avg = sum(event.valset_scores.values()) / len(event.valset_scores)
        print(f"Seed score: {avg:.2%}")

    def on_iteration_start(self, event):
        print(f"Iteration {event.iteration}, parent score: {event.parent_score:.2%}")

    def on_reflection(self, event):
        for comp, text in event.proposed_texts.items():
            print(f"Proposing for {comp}: {text[:100]}...")

    def on_valset_eval(self, event):
        if event.is_new_best:
            print(f"NEW BEST: {event.valset_score:.2%}")

    def on_optimization_complete(self, event):
        print(f"Done! Best: {event.best_score:.2%} in {event.total_iterations} iters")

# Use with other observers - they all work together
optimizer = GEPA(
    metric=my_metric,
    auto="medium",
    server_url="http://localhost:3000",
    observers=[MyObserver()],
    verbose=True,  # Also keep LoggingObserver
)

Migration Checklist

For Notebooks

Install: pip install dspy-gepa-logger
Change import: from gepa_observable import GEPA
Add server_url for dashboard
Add custom observers if needed
Run cells in order - API is 100% compatible

For Scripts

Install: pip install dspy-gepa-logger
Find all from dspy.teleprompt import GEPA
Replace import
Add observability parameters to GEPA constructor
No other changes needed

References

references/api-reference.md - Complete parameter docs, observer protocol, all event types
references/examples.md - Full before/after code examples