Modular Prompt Optimization (2026)

Free

Treats prompts as structured objects; optimizes each semantic section independently with local textual gradients

FreeFree tier

Inputs: textOutputs: text

Type

Open Source

Visit Website GitHub (40 stars)

About Modular Prompt Optimization (2026)

Modular Prompt Optimization (MPO) is a schema-based prompt optimization framework that treats prompts as structured objects composed of fixed semantic sections—system role, context, task description, constraints, and output format. It applies section-local textual gradients, generated by a critic language model, to refine each section independently while keeping the overall prompt schema fixed. Updates are consolidated through de-duplication to reduce redundancy and interference between components, yielding an interpretable and robust optimization process. MPO is evaluated on two reasoning benchmarks (ARC-Challenge and MMLU) using LLaMA-3 8B-Instruct and Mistral-7B-Instruct as solver models, consistently outperforming untuned structured prompts and the TextGrad baseline without modifying model parameters or altering prompt structure.

Key Features

Treats prompts as structured objects with fixed semantic sections (system role, context, task description, constraints, output format)

Applies section-local textual gradients generated by a critic language model

Refines each section independently while keeping the overall prompt schema fixed

Consolidates updates through de-duplication to reduce redundancy and interference

Interpretable and robust optimization process without modifying model parameters

Designed for smaller open-source instruction-tuned models (e.g., LLaMA-3 8B-Instruct, Mistral-7B-Instruct)

Evaluated on reasoning benchmarks (ARC-Challenge, MMLU) with consistent accuracy gains

Pros & Cons

Pros

Outperforms untuned structured prompts and the TextGrad baseline on reasoning benchmarks
Maintains a fixed prompt schema, preserving critical instructions and preventing uncontrolled prompt growth
Interpretable optimization process with section-wise refinement and deduplication
Requires no modification to model parameters or changes to prompt structure
Effective for small open-source instruction-tuned models that rely on explicit structure

Cons

Only evaluated on two reasoning benchmarks (ARC-Challenge, MMLU); generalizability to other tasks not yet demonstrated
Requires a separate critic language model to generate textual gradients, adding computational overhead
Relies on a predefined prompt schema; may not be suitable for tasks where optimal schema is unknown or varies
Untested on creative or open-ended tasks that do not fit a fixed semantic section structure

Best For

Improving reasoning performance in small open-source language modelsAutomatic prompt optimization for structured tasks with explicit requirementsControlling LLM behavior, reliability, and reasoning via optimized structured prompts

FAQ

What is Modular Prompt Optimization (MPO)?

MPO is a schema-based framework that treats prompts as structured objects with fixed semantic sections and optimizes each section independently using section-local textual gradients generated by a critic language model.

How does MPO differ from existing prompt optimization methods?

Unlike methods that treat prompts as monolithic blocks, MPO maintains a fixed prompt schema and refines each section locally, preserving critical instructions and preventing uncontrolled prompt growth.

What models and benchmarks were used to evaluate MPO?

MPO was evaluated on ARC-Challenge and MMLU using LLaMA-3 8B-Instruct and Mistral-7B-Instruct as solver models.

How does MPO compare to the TextGrad baseline?

MPO consistently outperforms the TextGrad baseline across both benchmarks and models, achieving substantial accuracy gains without modifying model parameters or altering prompt structure.