GPT

Automatic Optimization

Automatic Optimization | Paper | Key Contribution | |-------|-----------------| | [ProTeGi / Gradient Descent for Prompts (2023)](https://arxiv.org/abs/2305.03495) | Textual gradient descent — source paper for many auto

Prompt text

Original English text. Replace placeholders with your own details.

### Automatic Optimization

| Paper | Key Contribution |
|-------|-----------------|
| [ProTeGi / Gradient Descent for Prompts (2023)](https://arxiv.org/abs/2305.03495) | Textual gradient descent — source paper for many auto-optimization methods |
| [DSPy (2023)](https://arxiv.org/abs/2310.03714) | Prompts as compilable programs — defines the engineering-first paradigm |
| [MIPRO / Multi-Stage DSPy (2024)](https://arxiv.org/abs/2406.11695) | Optimizes instructions and demonstrations across multi-stage LM programs |
| [TextGrad (2024)](https://arxiv.org/abs/2406.07496) | "Autograd for text" — LLM feedback as gradients, published in Nature |
| [GEPA (2025)](https://arxiv.org/abs/2507.19457) | Reflective evolution outperforms GRPO by 6–20 pts with fewer rollouts |
| [Modular Prompt Optimization (2026)](https://arxiv.org/abs/2601.04055) | Treats prompts as structured objects; optimizes each semantic section independently with local textual gradients | [PDF](https://arxiv.org/pdf/2601.04055) |
| [Causal Prompt Optimization (2026)](https://arxiv.org/abs/2602.01711) | Reframes prompt design as causal estimation — uses Double Machine Learning to isolate prompt effects | [PDF](https://arxiv.org/pdf/2602.01711) |
| [Self-Evolving Memory for Prompt Optimization (2026)](https://arxiv.org/abs/2603.21520) | Memory-augmented APO that stores historical refinement insights and reuses them across iterations | [PDF](https://arxiv.org/pdf/2603.21520) |
| [Combee: Scaling Prompt Learning for Self-Improving Agents (April 2026)](https://arxiv.org/abs/2604.04247) | Berkeley/Stanford (Stoica, Zou, Gonzalez): scales parallel prompt learning with up to 17x speedup over ACE/GEPA via parallel scans and dynamic batching; evaluated on AppWorld, Terminal-Bench, FiNER | [PDF](https://arxiv.org/pdf/2604.04247) |
| [REprompt: Prompt Generation for Intelligent Software Development Guided by Requirements Engineering (Jan 2026)](https://arxiv.org/abs/2601.16507) | Multi-agent prompt optimization framework that applies requirements engineering (elicitation, analysis, specification, validation) to generate production-ready system and user prompts for agent-based software development | [PDF](https://arxiv.org/pdf/2601.16507) |
| [Self-Distillation Improves Code Generation (April 2026)](https://arxiv.org/abs/2604.01193) | Apple: embarrassingly simple self-distillation (SSD) — sample from model, fine-tune on raw unverified samples via cross-entropy; no reward model, no verifier, no RL; Qwen3-30B 42.4% → 55.3% pass@1 on LiveCodeBench v6; gains concentrate on hard problems; open source | [PDF](https://arxiv.org/pdf/2604.01193) |
| [SePO: Self-Evolving Prompt Agent for System Prompt Optimization (June 2026)](https://arxiv.org/abs/2606.04465) | NUS/CityUHK: closes the self-referential loop by treating the prompt agent's own system prompt as an optimization target alongside task-agent prompts; open-ended evolutionary search with an archive of stepping-stone candidates; two-stage pre-train/fine-tune pipeline generalizes to held-out tasks; +4.49 points over Manual-CoT on AIME'25, ARC-AGI-1, GPQA, MBPP, Sudoku | [PDF](https://arxiv.org/pdf/2606.04465) |

Source & attribution

Awesome Prompts · Awesome Prompts

Source-listed license: GPL-3.0. Prompt text is preserved; titles and previews may be shortened for display. Source examples are not NexGateHub test results.

View original source ↗ · Source credits & licenses

Explore more prompts →