The agent harness becomes part of model optimisation
Hugging Face has published a large-scale experiment showing that a model can be adapted to the exact software harness in which it will operate. The team trained separate LoRA adapters for Qwen3.8-27B under Pi, OpenCode and Hermes Agent, then evaluated them inside those same agent loops. The premise is important: a general model is not automatically efficient in every harness because tool schemas, context handling and interaction patterns differ. Training with the deployment harness in the loop lets reinforcement learning optimise not only whether a task is solved, but how the agent reasons and calls tools in that environment.
A validated task pipeline supports the training
The project assembled agentic coding tasks from SWE-smith, SWE-Gym and Tasksmith-generated work based on merged pull requests. Tasks passed licence and metadata screens, duplicate and benchmark-contamination checks, semantic validation and difficulty filtering. Hugging Face reports that 858 of 1,127 Hub candidates passed the initial validation stage, while 171 tasks occupied the useful middle ground where the base model sometimes succeeded and sometimes failed. That difficulty screen matters for group-relative policy optimisation because tasks that always pass or always fail provide little comparative learning signal.
Training runs inside the actual agent software
Rather than reimplementing an agent loop inside a reinforcement-learning framework, the experiment uses the target harness as shipped. A proxy records the tokens generated between the harness and a vLLM server, while TRL’s asynchronous GRPO trainer updates the adapter. The final training mixture included 750 harness-adapter environments and 300 Termgrade tasks. Hugging Face says the three runs consumed 167 million trained tokens across 598 cumulative steps. Training against the real loop reduces mismatch between development and deployment, although version changes in a harness can still alter behaviour after an adapter is produced.
Specialists improve accuracy in their own harnesses
Each specialist adapter beat the base Qwen3.8-27B model under its target harness on the reported SWE-bench Pro hard subset and Terminal-Bench 2.1 evaluations. Under Pi, the SWE-bench Pro result increased from 39.2% to 64.7%. Hermes rose from 29.4% to 43.1%, while OpenCode moved from 35.3% to 51.0%. Terminal-Bench gains were also positive. These are paired experimental results, not universal guarantees. They show that harness-specific reinforcement learning can recover substantial performance, but production teams should repeat evaluations with their repositories, tools, prompts and failure costs.
Efficiency gains may matter as much as benchmark scores
On tasks solved by both the base and adapted models, Hugging Face reports 56% to 72% fewer tool calls on SWE-bench Pro and 53% to 70% fewer on Terminal-Bench 2.1. Shorter trajectories can reduce latency, token consumption and the number of opportunities for an unsafe or incorrect action. The team also observed stronger first-attempt behaviour rather than simply longer reasoning. Efficiency must be measured carefully: fewer calls are valuable only when task completion remains correct, and a terse agent that skips verification may look cheap while increasing downstream repair work.
A merged super-adapter trades specialisation for portability
The three LoRA updates were close to orthogonal, allowing the team to concatenate them into a rank-96 super-adapter. A half-strength weighting delivered the best balance in the selection experiments. The merged adapter improved over the base model across all three harnesses, though it did not preserve every specialist gain. With thinking enabled, it reportedly beat or tied the thinking-enabled base model on the evaluated harness and benchmark combinations. This creates a practical choice: deploy a specialist when one harness dominates, or accept some lost peak performance for a single adapter that can operate across several agent environments.
Open artifacts make the claims more inspectable
Hugging Face released adapters, GGUF files, reinforcement-learning tasks, trajectories and supporting plugins through the Hub. The article also documents filtering, learning-rate experiments, task composition and paired results. That level of disclosure helps other teams reproduce or challenge the work instead of relying on a headline benchmark. Reproduction still requires significant compute and care. The reported setup used NVIDIA B200 GPUs, vLLM serving and fresh sandboxed trials. Organisations should also assess the licences and provenance of training tasks before adapting the recipe to commercial or sensitive repositories.
What platform teams should take from the experiment
The central lesson is that an agent’s harness is not neutral plumbing. It shapes the context a model sees, the tools it can call and the sequence through which work is completed. Those characteristics can be targeted during training. For teams operating coding agents at scale, a small adapter may offer a more controllable optimisation path than replacing the base model whenever a benchmark moves. The responsible route is to build representative tasks, validate verifiers, track tool-call safety and compare total completed-task cost. Harness adapters are promising precisely because they connect model improvement to a measurable production environment.