A fine-tuned Qwen2.5-7B model beat Claude Sonnet 3.7 - one of the best frontier models available - 88% to 31% on a narrow power-outage classification task. That number is real, and it should make any team pause before assuming a bigger model and a better prompt will always win.
The fine-tuning versus prompt engineering debate has run for years, but in the last few months two concrete results have landed that actually change the calculus. One of them favors fine-tuning on narrow tasks. The other suggests prompt optimization has been dramatically underestimated as an alternative to reinforcement learning. Both are worth understanding before you commit time or budget.
When a 7B model outscores a frontier model
In a 2025 study from Highlighter.ai, a fine-tuned Qwen2.5-7B went up against Claude Sonnet 3.5 and 3.7 with prompt engineering. The fine-tuned 7B model hit 88% accuracy on power-outage classification versus 31% for prompted Claude.
That gap is not a rounding error. It is the signature of a task so narrow that a frontier model's general knowledge actively works against it. Claude 3.7 is trained on everything; the fine-tuned Qwen2.5-7B is trained on exactly this domain, in exactly this format, with exactly these labels.
On some reasoning and summarization tasks, strong prompt design can rival or beat fine-tuned models. On specialized domains - domain-specific code generation, tightly scoped classification - fine-tuned models often significantly outperform prompt-only setups. The key word is specialized. When the output space is small, the label distribution is stable, and you have a few hundred clean examples, fine-tuning is worth the overhead. When those conditions aren't met, you're paying for a fine-tuned model that will drift the moment the task changes.
Prompt edits are free and instant; weight edits are expensive and slow. That asymmetry matters for any team that is still figuring out what the task actually is. Lock in a fine-tuned model too early and you've encoded your current misunderstanding into the weights.
What GEPA actually showed about prompt optimization
The more surprising result came from ICLR 2026. GEPA was introduced at ICLR 2026 as a gradient-free optimizer that uses natural language reflection rather than policy gradients to adapt prompts. Built on the DSPy framework, it outperforms reinforcement learning methods like GRPO by up to 20% while using 35x fewer model rollouts.
That comparison deserves some unpacking. GRPO - Group Relative Policy Optimization - often requires thousands of rollouts to learn new tasks. GEPA replaces the gradient signal with language. The key insight is that natural language provides a much richer learning signal than scalar rewards. Instead of "this output scored 0.7," GEPA sees "this output failed because the model confused temporal references with causal claims." Unlike RL-based approaches that require thousands of rollouts, GEPA's reflection-based approach means you can optimize prompts with 20-100 examples.
Across six tasks, GEPA outperforms GRPO by 6 percentage points on average and by up to 19pp, while using up to 35x fewer rollouts.
GEPA also outperforms the leading prompt optimizer, MIPROv2, by over 10% - including +12% accuracy on AIME-2025.
This is not a toy result. GRPO is what teams reach for when they want to adapt a model to a new task without full supervised fine-tuning. GEPA argues you can often get further, faster, by teaching the model to reflect on its own failures in language - and then updating the prompt rather than the weights.
In 2026, the highest-performing systems use both, in a specific order. Start with prompt optimization - DSPy and GEPA - which now beats GRPO by 6 to 19 points using 35x fewer rollouts. Move to SFT only when you need a smaller model or a frozen behavior.
That sequencing is the practical takeaway. It is not a philosophical argument about which technique is superior. It is a workflow: exhaust prompt optimization first, because it is cheaper and reversible, then decide whether the remaining gap justifies the weight of a fine-tuning run.
What the full-context trap costs you
There is a parallel trap in agent memory that echoes the same mistake. The full-context approach - dumping complete conversation history into the prompt - delivers the highest accuracy ceiling, but at a cost that makes it categorically unusable in production. Scaling context is not the same as scaling capability.
Unlike humans, who dynamically integrate new information and revise outdated beliefs, LLMs effectively "reset" once information falls outside their context window. Even as models like GPT-4 (128K tokens), o1 (200K context), Claude 3.7 Sonnet (200K tokens), and Gemini push the boundaries of context length, these improvements merely delay rather than solve the fundamental limitation.
The same logic applies to fine-tuning versus prompting. Bigger is not always better. More parameters is not always better. What matters is fit: does the approach match the structure of the task?
The two largest gains in Mem0's new retrieval algorithm are on temporal queries (+29.6 points over the old algorithm) and multi-hop reasoning (+23.1 points). These are the two categories that most directly reflect how agents handle real user histories, in which facts accumulate, change, and relate to one another over time. That is a domain-specific optimization - not a general one - and it required understanding exactly where the baseline approach was failing.
The same principle holds when you are deciding whether to fine-tune an LLM for a workflow in Slack or Teams. If the task is tightly scoped - classify this support message as billing vs. technical vs. refund - a fine-tuned small model will likely beat a prompted frontier model on accuracy and cost. If the task is broad or evolving - summarize this thread, identify the next action, draft a reply in our tone - prompt optimization is where you should start. A teammate like Beagle would surface structured outputs from those conversations precisely because the task is defined enough to optimize around, without committing to weights that will need to change when the workflow does.
The decision is not binary
The fine-tune-or-prompt question is wrong. The real question is sequencing.
Start with prompt engineering. It takes hours, not weeks. If you can get to 80% of your accuracy target with a well-structured prompt and a few-shot example set, you have learned what your task actually requires. That knowledge makes any subsequent fine-tuning run faster and cheaper, because you are no longer guessing what the model needs.
The practical takeaway is not to copy benchmark numbers blindly, but to test whether your own trace quality is high enough for reflection-driven optimization to work. GEPA works best when you have rich failure traces - tool calls, reasoning chains, structured outputs - that an LLM can inspect and learn from. If your traces are thin or your scoring function is vague, the reflection signal degrades and you are back to needing labels.
Use fine-tuning to encode foundational domain knowledge. Use prompt engineering to adapt that foundation to specific tasks, users, or workflows. This combination improves accuracy, keeps prompts simple, and can reduce inference cost by avoiding excessively long inputs.
That is the practical architecture. Not fine-tuning or prompting. Fine-tuning for what is stable, prompting for what moves.