AI Coding Agent Fine-Tuning on Private Codebases

A developer asks an AI coding agent to write a validation function. Then a teammate opens the pull request and winces: wrong variable naming, wrong error-handling style, a module structure that looks like it wandered in from a different company. The function works. It's still wrong. That gap is the whole story behind why generic coding agents stumble inside private codebases, and it has nothing to do with the model being bad at programming.

It has to do with the model never having seen how this particular team writes code. Ai2, releasing its SERA coding agents, put the underlying issue this way: closed models haven't seen an organization's internal code, so they don't know its custom data pipelines, its internal APIs, or its specific conventions. A model trained on the open internet knows programming in general. It doesn't know that this company names its validators validate_user instead of check_user_input, or that error handling here always logs to a specific internal service before raising an exception.

Call it a domain gap, not a competence gap. The agent isn't hallucinating facts or making logical errors. Every generated function, every suggested refactor, every drafted pull request carries the wrong dialect. Multiply that across a team of developers and the agent stops saving time: it just shifts the work from writing code to correcting code, which is a worse trade than not having the agent at all.

The failure is behavioral, not factual, and that changes what can fix it

The mistake many teams make here is assuming this is a lookup problem: give the model access to the internal docs, the internal API references, the style guide, and the agent will fall in line. The gap described above isn't about missing facts; it's about missing behavior.

A retrieval-augmented generation (RAG) system can hand the model the exact right internal API reference, at the exact right moment, and the model can still write the response in a naming convention nobody on the team uses. It doesn't touch "how does the model generate." Those are separate problems, and the behavioral one is the one breaking things in production.

Four specific behaviors expose where prompting and retrieval run out of road. Structured output reliability is one: prompt-only setups still hallucinate fields in JSON or YAML outputs, even with a schema pasted directly into the prompt. Consistent output format rounds out the list: style drifts from session to session because nothing is anchoring it.

What these four have in common is that they're not solved by finding the right document. They're solved by changing how the model generates in the first place, and that only happens when the behavior gets embedded into the model's weights directly. That's the piece prompting and retrieval structurally cannot deliver, no matter how good the retrieval index gets. It's also the exact job fine-tuning is built to do.

What fine-tuning on private code does mechanically

Fine-tuning takes a model that already knows how to program and updates it using examples drawn from the organization's own code, so the model picks up the organization's dialect. The training material is structured input-output pairs, not raw source files dumped into a folder: a function paired with its docstring, a bug report paired with the commit that fixed it, an internal API call paired with the output it's expected to produce.

For example, a company has an internal Python utility library with functions like validate_email and validate_user. The model doesn't memorize these functions. It absorbs the pattern underneath them, and the next time it's asked to write something similar, it reaches for the same shape.

Getting that training data ready is the part of the process that determines whether the whole project works. Inconsistent formatting and deprecated code need to come out too, because the model can't tell a good example from a bad one. It just learns whatever pattern sits in front of it, including the organization's worst habits if those habits made it into the training set.

Full fine-tuning updates every parameter in the model and is almost never the right call. There's also a distillation path: train a smaller model on the outputs of a working large-model pipeline, compressing frontier-level behavior into a model cheap enough to run constantly.

The cost of private-codebase specialization in 2026

The compute cost of this kind of specialization has dropped enough that it's no longer a large-lab project. Small teams and individual researchers can now run it.

Ai2's SERA release, from January 2026, is the clearest evidence of that floor. Reproducing the performance of the previously best open-source coding model costs a fraction of what it costs to reach performance on par with leading industry models of comparable size, with training taking 40 GPU days or fewer on a cluster of 2 NVIDIA Hopper GPUs or NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs. That's a training run a small team can plan around, not a budget line that needs a large lab behind it.

Part of what makes SERA notable for private codebases specifically is its Soft-Verified Generation method, which operates entirely independent of a repository's unit tests. Most internal codebases don't have clean, complete test suites sitting around waiting to validate training data. A method that doesn't depend on unit test coverage to function is a method that actually works on the kind of messy, real internal code most organizations have, rather than the tidy open-source repos most benchmarks are built from.

None of this means fine-tuning is free. GPU time is the cheap part of the bill. Building and cleaning the training dataset, running the evaluation loop, and managing version control across model iterations is what costs money. That's where project budgets actually get spent, and it's where projects actually stall. SERA shows the compute floor has dropped. It doesn't mean fine-tuning projects are cheap across the board; it means the compute piece has stopped being the bottleneck it used to be.

Fine-tuning versus retrieval-augmented generation

Fine-tuning and RAG aren't competing for the same job. They solve two different halves of the private-codebase problem, and most production systems end up needing both running at once.

The split comes down to what's stable versus what changes. RAG handles fresh, specific facts: current documentation, a live API response, a schema that got updated last week. That information needs to stay current, and it needs to stay auditable. Fine-tuning handles the behavioral patterns that don't shift commit to commit: naming conventions, output formats, the organization's coding idioms. Those patterns are stable enough to bake into weights, because they aren't going to change next Tuesday.

That auditability point carries real weight in compliance-heavy settings. A fine-tuned model can't point to the document that justified a given answer, because the knowledge is distributed across its weights. A retrieval system can show exactly which document it pulled from. For teams where auditors need to trace a code suggestion back to a source, that asymmetry alone can rule a pure fine-tuning approach out.

The strongest check on fine-tuning's necessity comes from a comparative study run with Tencent's WXG department, using more than 160,000 internal C++ files. RAG, implemented with the right retrieval methods (BM25 lexical search outperformed dense embedding models in this study specifically), achieved higher accuracy than fine-tuning alone. That result deserves to be taken at face value rather than explained away: fine-tuning is not automatically the superior approach for every private codebase, and retrieval done well can outperform it.

The practical sequence most teams should follow runs: prompt first, then RAG, then fine-tune, then distill if volume justifies it. Fine-tuning earns its spot in the stack only after retrieval has been tried and its limits on behavioral alignment have actually shown up. If the problem is stable behavior the model needs to produce consistently, fine-tuning is the right tool. If the problem is knowledge that changes often and needs to stay traceable, RAG is the right tool. The hybrid of both, RAG for facts and fine-tuning for behavior, is becoming the standard production setup for teams that take private-codebase performance seriously.

The security and IP risks that fine-tuning on private code introduces

Fine-tuning on private code creates a risk category distinct from ordinary day-to-day use of an AI tool, because the resulting model's weights now carry organizational knowledge in a form that's hard to inspect and even harder to remove once it's in there.

Training data can contain credentials. Models fine-tuned on internal repositories have been shown to regurgitate old but still-valid API keys, internal URLs and IP addresses, and hardcoded tokens that were sitting in the training corpus. Stripping these out before training prevents a model from leaking secrets the moment someone asks it the right question.

Corpus poisoning is a real attack surface too. Once those patterns get embedded into the model's weights, they're hard to spot and expensive to remove.

The Samsung incident from 2023 remains the field's reference case. Internal source code was disclosed through a consumer LLM, and the fallout led to a company-wide ban on consumer-facing LLMs across its semiconductor division. That event is why demand for on-premise training pipelines exists at all: the risk of data walking out the door isn't hypothetical, it already happened to a company with Samsung's resources.

Cycode's 2026 State of Product Security research adds a second layer to the concern. Every organization surveyed had AI-generated code somewhere in its codebase, but only a small minority reported full visibility into how AI gets used across development. That visibility gap matters directly here: if a team doesn't know where AI-generated code has already landed, it can't know what a fine-tuned model has already absorbed from it.

Cloud-hosted training pipelines raise a question about data exposure that's worth asking directly. Compliance teams are increasingly asking that question before signing anything. The mitigation path is straightforward even if the execution isn't: strip sensitive material before training, lock down access to the resulting weights, keep audit logs of where the training data came from, and treat the fine-tuned model itself as a high-value asset deserving the same protection as the codebase it came from.

The jurisdictional wall regulated industries face around cloud-based fine-tuning

For regulated industries, the obstacle to cloud-based fine-tuning is jurisdictional, and the EU AI Act's maturing requirements are tightening that wall.

Regulators are asking where the data physically sits during training, and who can access the gradients and intermediate checkpoints along the way. Those are infrastructure questions, not security questions, and they don't have a contractual answer.

A US-hosted fine-tuning API with regional endpoints can fail a data-flow review even when its primary compute sits in the correct region. Usually it's that logs, telemetry, and checkpoints route through global infrastructure behind the scenes, so the data moves internationally even when the headline compute doesn't.

The practical fallout lands on specific sectors: EU-based financial institutions, healthcare networks operating under strict data-residency rules, and Singapore financial institutions working under MAS frameworks. All of them are shut out from most cloud fine-tuning APIs for any code that touches regulated data, because the infrastructure itself can't satisfy the review.

Self-hosted fine-tuning infrastructure, running the entire training and inference stack inside the organization's own Kubernetes cluster with outbound traffic cut off entirely, is the architectural answer to that constraint. It's the only setup that actually satisfies what regulators are asking about jurisdiction over the data. Platforms built for private deployment have started treating on-premise fine-tuning as a first-class feature rather than an afterthought, specifically because this constraint isn't going away for regulated sectors.

The operational costs that persist after the initial fine-tune

The one-time compute cost of fine-tuning turns out to be the cheapest part of the whole project. Keeping a fine-tuned model aligned with a codebase that keeps changing is ongoing work most teams underestimate, and it never fully stops.

Base-model drift is one recurring issue. A developer just starts noticing that suggestions feel worse than they used to, with no clear trigger pointing to why.

Adapter versioning and rollback is another. A fine-tuned model is a versioned artifact, and when a retrain introduces a regression, the team needs a way to roll back to the previous version. That requires the same versioning discipline teams already apply to production software, and most teams haven't set that discipline up before their first fine-tuning project gets underway.

Retraining cadence matters because codebases that evolve quickly, picking up new internal APIs, deprecating old modules, shifting conventions, will outpace a model trained on a single snapshot in time. Quarterly revalidation is a reasonable floor for most teams. Teams shipping fast may need to retrain more often than that.

Catastrophic forgetting is a risk to track across every retrain cycle, not just the first one. Parameter-efficient methods reduce this risk but don't erase it, so evaluation has to include general coding benchmarks alongside domain-specific tests every time.

Overfitting to training data rounds out the list. The standard rule for supervised learning applies here with full force: garbage in, garbage out.

None of this is a reason to skip fine-tuning. Teams that plan for this cycle from day one tend to find it manageable. Teams that treat fine-tuning as a one-time task tend to find themselves back at square one within a quarter.

Building the full private-codebase fine-tuning stack

The real decision facing most teams is which combination of base model, training method, deployment setup, and retrieval layer actually fits a given codebase, regulatory situation, and operational bandwidth.

Full fine-tuning is almost never the right place to start.

On deployment architecture, the choice between a cloud-hosted fine-tuning API and self-hosted infrastructure comes down mostly to regulatory posture.

On the agent platform running all of this, the fine-tuned model needs somewhere to live that gives teams visibility and guardrails over its behavior at scale. Platforms offering sandboxed execution and native repository integrations can cut down the friction of pulling and securing that training material from a live codebase while keeping an audit trail of what data went in, which is the gap OpenHands, an open-source platform for deploying autonomous coding agents, addresses through sandboxed VPC execution and integrations with GitHub, GitLab, and Jira.

On the retrieval layer, fine-tuning handles stable behavioral knowledge while retrieval handles current facts, fresh documentation, and anything that needs to stay auditable. They're complements to each other, not substitutes.

None of this is a project with a finish line. The fine-tuned model, the retraining pipeline, the evaluation suite, and the agent platform running it all together form one system that the organization ends up maintaining indefinitely, the same way it maintains the codebase the model was trained on in the first place.