Open-Source LLM Selection for Coding Agents

Picking an open-source model for a coding agent is about architecture, licensing, hardware fit, and how the model behaves once it's wrapped in an agent scaffold — even though half the industry still treats it as a benchmark-reading exercise. Leaderboard screenshots tell you which bar is tallest; they don't tell you which model to actually deploy.

First, a quick vocabulary check, because the term "open source" gets abused constantly in this space. True open source means the weights, the code, the training data, and a real license are all available. "Open weight" means you get the weights and maybe some code, but the training pipeline stays locked in a vault somewhere. Most of the models people call "open-source LLMs" are actually open-weight. That distinction matters if your legal team ever asks how a model was trained, or whether you can reproduce it.

For coding agents, this isn't pedantry. Open-weight access means you control the deployment environment, you can fine-tune on your own codebase, nothing leaves your servers, and you get to pick a different underlying model depending on the task at hand. Open-weight status is a starting point for evaluation, though, rather than a selection criterion on its own. Everything below is what actually separates one model from another once you're running it in production.

How the dominant architecture — Mixture-of-Experts — changes what deployment actually costs

Dense transformers fire every parameter for every single token. Mixture-of-Experts (MoE) models route each token through only a slice of the total network, which means a model with hundreds of billions of parameters can run at a fraction of the compute cost you'd expect from its size alone.

By mid-2026, MoE had become the default architecture across pretty much every serious open-weight release. The top models by both benchmark score and community adoption are all built this way.

The numbers make the point better than any explanation. DeepSeek V4 Pro carries 1.6 trillion total parameters, but only 49 billion of them activate per token. GLM-5.2 sits at 744 billion total, with roughly 40 billion active, a 5.4% activation rate. That gap between "total" and "active" is the whole story.

So the size number in a model's name tells you almost nothing about what it costs to serve. Teams that compare models by total parameter count are answering a question nobody asked. Hardware planning and cost estimates need to start from the active parameter figure, full stop.

This ratio drives home nicely with Qwen3-Coder-Next: 80 billion total parameters, only 3 billion active. That's what makes edge deployment, laptops, single-GPU boxes, possible for a model that on paper looks far too big for the job.

Architecture sets the cost floor. Whether the model can actually do agentic work is a separate question entirely.

What context window size and long-horizon capability actually mean for agent tasks

A one-shot coding assistant looks at a prompt and a file, then hands back an answer. A coding agent works across an entire session: tool outputs pile up, repo state shifts, test results and error logs and prior failed attempts all stack into the context window. That context keeps growing the whole time the agent is working.

A 1-million-token context window is table stakes now among the leading models, DeepSeek V4 Pro, GLM-5.2, MiniMax M3 through MSA. Hitting that number in a spec sheet and holding together reliably across a long agent session are two very different claims, though.

MiniMax M3 gives the most concrete evidence in this category. In internal testing, it reproduced an ICLR paper autonomously over roughly 12 hours, 18 commits, 23 experimental figures. In a separate test, it optimized a CUDA kernel over about 24 hours across 147 benchmark submissions, pushing hardware utilization from 7.6% up to 71.3%, a 9.4x speedup, while most other models stalled out within their first 30 submissions.

Kimi K2.6 takes a different angle on multi-step reliability. It can break a complex task into parallel subtasks, coordinating up to 300 sub-agents across 4,000 coordinated steps at once, which matters a lot if you're thinking about multi-agent orchestration down the line.

When a vendor tells you their context window is huge, the real question is whether the model stays coherent across it. Does it keep track of tool calls and prior state through the second half of a long session, or does it quietly fall apart? A published context size marks a ceiling, and what happens underneath that ceiling is a separate question worth testing directly.

Reading SWE-bench scores without being misled by the harness gap

SWE-bench Verified tests models against real GitHub issues pulled from popular Python repos, and the model has to resolve them end-to-end. That makes it the right category of benchmark for this decision, since it measures agentic engineering ability rather than autocomplete quality.

Here's the catch nobody puts on the marketing slide: vendors run their own tuned agent harness and report those numbers. The gap between a vendor's reported score and what you'd get running the same model through a standardized harness typically runs 10 to 30 points. Most of that swing comes from context retrieval and tool-use quality in the harness itself, not from anything different about the underlying model.

Terminal-Bench 2.0 shows the same problem even more sharply. The same model can swing 30 to 50 percentage points depending on which harness wraps it. The scaffold matters as much as the model does, sometimes more.

OpenAI actually stopped publicly reporting SWE-bench Verified scores in early 2026, partly because the distance between "scores well" and "is actually useful" had gotten too wide to responsibly ignore.

The right question when you look at any leaderboard isn't which model won. It's which harness ran it, which tools were available, what the retry policy looked like. SWE-bench Pro tries to close part of this gap: 1,865 tasks across 41 professional repos, built from GPL-licensed code specifically to deter training-data contamination. It's a harder signal, and a more trustworthy one.

DeepSWE pushes further in the same direction: 113 original tasks, with prompts that demand far more code and output than typical benchmarks. That's closer to what real engineering work actually looks like, a short bug report dropped into a sprawling codebase, with a fix that touches five files you didn't expect. Only closed models have reported scores on it so far, but it's worth watching as more labs run it.

Bottom line: treat benchmark scores as a relative ranking inside one consistent harness, not as a hard prediction of how a model will perform on your stack.

Where the leading open-weight models actually land on the benchmarks that matter

With that caveat firmly in place, the actual numbers still tell you something useful.

On SWE-bench Verified, vendor-reported scores put DeepSeek-V4-Pro-Max at 0.806, good for rank 8 overall and the top open-weight entry by this measure. MiniMax M3 comes in at 80.5%, Qwen3.7 Max at 80.4%. Devstral 2, built specifically for agentic software engineering at 123 billion parameters, lands at 72.2%. Its smaller sibling, Devstral Small 2 at 24 billion parameters, scores 68% and runs on a single RTX 4090 or a Mac with 32GB of RAM. For scale, Claude Fable 5 leads the overall leaderboard at 0.950 across 111 evaluated models, so the gap to the frontier is real, but it's shrinking.

SWE-bench Pro, the harder and contamination-resistant version, tells a slightly different story. GLM-5.2 leads open-weight entries in vendor-aggregate reporting at 62.1%, ahead of Qwen3.7 Max at 60.6%, MiniMax M3 at 59.0%, and Kimi K2.6 at 58.6%. Run the same models through Scale's standardized harness, though, and the top open-weight score drops to 38.7% for Qwen3-Coder-480B-A35B. That drop is the harness gap made visible in one number.

On Terminal-Bench 2.1, which tests agentic terminal use, GLM-5.2 leads at 81.0, relevant if your agent runs shell commands, kicks off test suites, or manages build pipelines. On Frontend Code Arena, Kimi K3 ranks first in frontend coding, scoring 93.4% on SWE-bench Verified through Vals AI's independent harness, though its weights were still pending full release at launch in July 2026.

No model wins across the board. The ranking shifts depending on whether you're testing repo-scale reasoning, terminal use, frontend code, or long autonomous runs. Benchmark position is one input among several, not the final answer.

How licensing terms translate into real deployment and commercialization constraints

Licensing decides whether you're legally allowed to ship the thing, well beyond being a formality.

DeepSeek V4 Pro uses MIT, about as permissive as it gets, with the fewest restrictions on commercial use and redistribution. Devstral Small 2 uses Apache 2.0, permissive and backed by a patent grant, generally safe for commercial products. Kimi K2.6 uses a modified MIT license that permits commercial use but adds an attribution clause, a specific obligation your legal team needs to actually read, not skim. MiniMax M3 ships under a MiniMax Community License with its own commercial-use conditions attached, and those details are not interchangeable with MIT or Apache, despite sounding similar.

Before picking a model, get answers to a short list of questions. Can you deploy it inside a product you sell to customers? Can you fine-tune it and redistribute the result? Does an attribution notice need to show up in your UI or your docs somewhere?

If fine-tuning on a proprietary codebase is the goal, Apache 2.0 and MIT give you the cleanest path. Community or modified licenses can attach conditions to anything you build on top, and those conditions vary enough between vendors that you can't assume one license behaves like another just because both get called "open."

The whole licensing landscape here is still shifting. What counts as "open source" in AI is genuinely contested, and some licenses that look permissive at a glance carry carve-outs for specific uses, serving at massive scale, or building a model that competes with the original. Treat the license as a technical dependency, reviewed at the same stage as a security check and well before deployment, rather than something bolted on afterward.

Matching model size and hardware requirements to what your infrastructure can actually run

The MoE section already made the core point: active parameters drive serving cost, not total parameters. Here's what that looks like in actual hardware terms.

At the small end, Devstral Small 2, 24 billion total parameters and purpose-built for coding agents, runs on a single RTX 4090 or a Mac with 32GB of RAM. That's genuine edge deployment, useful on a developer's own workstation or inside a CI pipeline with no GPU cluster in sight. Qwen3-Coder-Next, despite a larger total parameter count at 80 billion, stays just as viable on modest hardware because only 3 billion parameters activate per token.

Move up a tier and you hit Devstral 2 at 123 billion and Qwen3-Coder-480B-A35B, both of which need multi-GPU setups but stay within reach of a well-provisioned on-prem cluster. At the frontier, DeepSeek V4 Pro (1.6 trillion total, 49 billion active), GLM-5.2 (744 billion total, 40 billion active), and Kimi K2.6 (1 trillion total) all need serious infrastructure to self-host at production scale. For most organizations, API access is the more realistic path to using them.

Which brings up the tradeoff at the heart of this whole exercise. DeepSeek V4 Pro runs $0.435 per million input tokens and $0.87 per million output tokens through its API, appealing if you want the capability without building out the infrastructure. But going through an API brings back the exact data egress and vendor dependency concerns that probably pushed you toward open weights in the first place.

Latency also behaves differently for agents than for assistants. An agent grinding through a 4,000-step workflow can tolerate higher per-token latency than a tool that needs to autocomplete your next line of code in real time. That changes how you should actually calculate cost per useful output.

Before committing, test throughput at context lengths that match real agent sessions, not short demo prompts. Check how much latency degrades as context grows, and watch memory pressure under multiple concurrent sessions. Hardware that looks fine on a benchmark can buckle under five agents running at once.

What to evaluate when the model runs inside an agent framework rather than a prompt

A coding agent wraps the underlying model in a scaffold, tool definitions, retry logic, memory management, subagent orchestration, and what actually ships to your team is that whole wrapped package, evaluated on its own terms rather than judged by how the model performed sitting alone in a lab test.

A few things separate models once they're inside that wrapper, beyond whatever score they posted on a leaderboard. Does the model reliably produce valid tool calls under a defined schema, or does it hallucinate parameters and force you to build error-handling around its mistakes? Holding a coherent plan across 20 tool calls in a row is another marker worth checking, versus losing the thread and starting over from scratch. When a tool call fails or comes back with something unexpected, adjusting is what separates a strong model from one that loops on the same failed approach. And does it support real parallelism natively, the way Kimi K2.6 handles sub-agent decomposition, or does it need an external wrapper bolted on just to fake that behavior?

The agent scaffold itself carries as much weight as the model choice. Open-source frameworks like OpenHands, SWE-Agent, Agentless, AutoCodeRover, Moatless, and PatchPilot each handle context management, tool use, and retry policy differently, and the pairing between model and scaffold changes outcomes as much as swapping the model alone would. Pick the model first and ignore the scaffold, and you've only solved half the problem.