Mellum2.1's SWE-bench Verified score went from 2% to 47% between June and October, and JetBrains didn't change a single layer of the architecture to do it. The company released Mellum2.1 on October 8, 2026 as the next version of the 12B mixture-of-experts model it open-sourced in June. The architecture - a compact MoE with 2.5B active parameters under Apache 2.0 - is identical to Mellum2. What changed is everything that happens after pre-training. That gap between a frozen architecture and a radically different benchmark score is the interesting thing here, and it's worth understanding before you decide whether this model belongs in your stack.
What reinforcement learning in real environments actually means
Most RL post-training for coding models uses synthetic tasks or curated contest problems. JetBrains ran millions of sandbox tasks across thousands of environments. During training, the model invoked terminal and file-editing tools within real code repositories and received rewards only upon successfully passing tests. The reward signal is a test suite pass - not a judge model's preference, not human labels. Either the code works or it doesn't.
Mellum2.1 carries the same 2.5B active parameters as Mellum2, but adds extensive post-training reinforcement learning across thousands of sandboxed environments. The architecture is unchanged from Mellum2, but the model underwent extensive reinforcement learning in real sandboxed environments, improving its ability to explore codebases, edit files, and verify its own changes.
The practical consequence: Mellum2.1 handles agentic tasks much better than Mellum2. After millions of sandboxed runs in real environments during training, it explores a codebase, edits files, and checks its own changes. That's not a capability the base MoE had - the pre-training gave it language and code knowledge, but the RL taught it to act inside a repo.
Reading the benchmark table honestly
All models were evaluated by JetBrains with the same pipeline in thinking mode. All values are self-reported by JetBrains. That's the disclaimer in the model card, and it matters. There's no independent replication yet.
With that caveat stated, here's the full picture across the comparison set:
| Benchmark | Mellum2.1 | Mellum2 | Qwen3.5-9B | Gemma 4 E4B |
|---|---|---|---|---|
| SWE-bench Verified | 47.0% | 2.0% | 50.0% | 23.0% |
| SWE-bench Pro | 28.0% | 0.0% | 38.0% | 4.0% |
| LiveCodeBench v6 | 82.0% | - | 75.4% | 69.4% |
| Terminal-Bench 2.1 | 17.4% | - | 21.7% | - |
| HumanEval+ | 91.5% | - | - | - |
Mellum2.1 beats Mellum2 on 15 of 17 listed benchmarks and wins 5 of 17 against Qwen3.5-9B. The model scores best on LiveCodeBench and still trails Qwen3.5-9B on harder agentic software tasks and knowledge.
The pattern is clear: Mellum2.1 is competitive on structured coding tasks (LiveCodeBench, HumanEval+) but consistently behind Qwen3.5-9B on the harder agentic evaluations (SWE-bench Pro, Terminal-Bench). JetBrains' Apache-2.0 model adds repository-focused reinforcement learning, but its speed and capability comparisons come from the company rather than independent testing. Treat the numbers as directional until the community runs its own evals.
The throughput argument for local deployment
The case for Mellum2.1 in a self-hosted setup isn't just the benchmark scores - it's the inference economics. On one NVIDIA H200 under heavy load JetBrains reports it serves nearly 2x the tokens of Qwen3.5-9B, with multi-token prediction making single requests ~1.6x faster.
The first figure concerns throughput under load; it does not mean every individual response arrives twice as quickly. The second describes an MTP speedup and should not be rewritten as a 1.6-fold advantage over Qwen3.5-9B. Those are two different metrics - batch throughput and single-request latency - and they matter differently depending on whether you're running a coding agent that queues many parallel calls or serving one developer at a time.
The model runs via vLLM or SGLang on your own GPUs. GGUF builds start at 7.0 GB for llama.cpp, Ollama, and LM Studio. A 7 GB GGUF fits in the VRAM of a consumer 4090 with room left for context. That puts a coding agent model with a 131K context window on hardware a single engineer might already own.
Beagle in action#engineering, 10:22amWhere it fits - and where it doesn't
JetBrains targets three uses: agent worker, general reasoning assistant, and private self-hosted deployment. The third is the most defensible claim. Teams that can't send source code to a hosted API - financial services, healthcare, regulated government work - now have a genuinely capable option that runs on hardware they control under a permissive license.
For the agent worker role, the 47% SWE-bench Verified score puts Mellum2.1 in the range of models that are usable but not dominant. Qwen3.5-9B edges it on the hardest tasks at 50%. Per the model card, the model routes across 64 experts and activates 8 of them per token, spread across 28 transformer layers. The Thinking variant emits a visible chain-of-thought before producing its final answer. That reasoning trace is useful when you're debugging agent behavior - you can see where the model went wrong before it made an edit.
The weak spot is terminal-heavy tasks. Mellum2.1 scores 17.4 on Terminal-Bench 2.1, below Qwen3.5-9B at 21.7. If your agent needs to navigate shell environments, run system commands, or recover from terminal errors, the score gap is real.
Mellum2.1 open-weight coding agent model: common questions
What is Mellum2.1 and how is it different from Mellum2?
Mellum2.1 is a 12B mixture-of-experts open-weight coding model from JetBrains, released October 8, 2026. The architecture is identical to Mellum2 - 2.5B active parameters, 64 experts, 28 layers. The difference is post-training: Mellum2.1 was trained with reinforcement learning across millions of real sandboxed repository tasks, which lifted SWE-bench Verified from 2% to 47%.
Can Mellum2.1 run locally on consumer hardware?
Yes. GGUF-quantized builds start at 7 GB and run via Ollama, LM Studio, or llama.cpp. The 12B total parameter count with 2.5B active means memory pressure is lower than it looks on paper. A single 24 GB GPU handles the full-precision Thinking variant comfortably.
Is the Mellum2.1 license permissive for commercial use?
Mellum2.1 ships under Apache 2.0, which permits commercial use, modification, and redistribution without royalties. The weights are on Hugging Face. Note that open-weight does not mean the training data or training code are public - only the weights and the license are.
How do Mellum2.1's benchmarks compare to Qwen3.5-9B?
On JetBrains' self-reported evals, Mellum2.1 beats Qwen3.5-9B on LiveCodeBench v6 (82% vs 75.4%) but trails it on SWE-bench Verified (47% vs 50%), SWE-bench Pro (28% vs 38%), and Terminal-Bench 2.1 (17.4% vs 21.7%). For throughput under load on one H200, JetBrains claims close to 2x the token output of Qwen3.5-9B. None of these numbers have been independently reproduced yet.
What agent frameworks work with Mellum2.1?
The model exposes an OpenAI-compatible interface and is compatible with vLLM and SGLang for serving. JetBrains targets it as a sub-agent or worker role in multi-model pipelines. The Thinking variant's <think> output blocks need to be stripped before structured outputs or tool-call parsers read the response.