Two narrow models replace one general assumption
Exowatt has worked with AWS to build a repeatable pipeline for specialised models that support solar-energy operations. Its first use case triages alarms at sites that may have limited connectivity, while the second analyses market data for energy-dispatch decisions in the cloud. Both tasks require structured outputs and evidence tied to operational records rather than broad conversational ability.
For edge alarm triage, the team fine-tuned a Qwen3 1.7B model and quantised it to run offline on an existing CPU server. AWS reports a macro-F1 score of 0.911 on 320 held-out cases, above the frontier models tested for that narrow task. The absence of a metered inference service is described as zero incremental API or GPU cost, not zero ownership cost.
Serverless training creates repeatable deployment evidence
The cloud use case fine-tuned Qwen3 14B on five years of public ERCOT market data. Rule-labelled examples were expanded with paraphrases created by Claude Sonnet 4.6 on Amazon Bedrock, while deterministic checks rejected rewrites that changed labels, evidence or numbers. The final training set contained 86,281 rows, with separate validation and 2,000 held-out test cases.
AWS reports that the fine-tuned model reached 0.975 macro-F1 and 0.983 event accuracy, compared with 0.625 macro-F1 for Claude Opus 4.7 in the same benchmark. Estimated inference cost was US$0.60 per 1,000 answers, about 49 times lower than the cited Opus result. The merged model was imported into Amazon Bedrock Custom Model Import in about 14 minutes.
Citation and held-out tests support operational trust
The project used serverless SageMaker fine-tuning and evaluation, S3, Model Registry, Lambda, EventBridge and Bedrock. Training ran across 23 jobs and roughly 155 managed training hours over six days. The architecture matters because it preserves data, recipes, metrics and deployment artefacts, allowing the next specialised model to follow an established evidence path.
Small models can outperform general systems when the task, labels and output contract are tightly defined. They can also fail badly outside that boundary. Exowatt’s insistence that every claim cite an input row is a useful control, but operators still need drift detection, abstention rules and human escalation when a new fault family or market condition falls outside the training evidence.
What teams should validate before production
For implementation, teams should record the exact model, software, dataset, connector and policy versions behind every result. Representative evaluation must include normal work, boundary cases, deliberate misuse and component failure, with outcomes separated by user role, data sensitivity and consequence. A strong average should never hide a serious failure in a critical path. Changes should pass the same regression set before promotion, and operators need a documented fallback that preserves evidence rather than silently broadening access or lowering quality.
Procurement and planning should account for the human work required to review exceptions, maintain controls and investigate incidents. Published performance or efficiency claims are useful starting points, but production confidence comes from evidence collected in the organisation’s own environment. A named owner, measurable acceptance threshold, cost model and tested shutdown path turn a promising release into a service that can be governed over time. Where decisions affect safety, rights, money or regulated records, accountable professional review remains essential.
Security and operational evidence
Security review should map every identity, credential, network destination, stored artefact and administrative access path involved in the workflow. Logs need enough context to reconstruct who initiated an action, which evidence was consulted, what the system proposed and what ultimately executed. Sensitive prompts and traces should not be retained merely because they are useful for debugging. Retention, redaction and access must follow the underlying data classification, with a practical process for correcting or removing material when policy or law requires it.
Operational monitoring should measure quality and service behaviour together. Latency, throughput and availability matter, but so do unsupported claims, missed constraints, unsafe tool calls and the rate at which people overturn recommendations. Teams should rehearse dependency outages and partial failures before unattended operation. If a model, connector or external service becomes unavailable, the workflow should fail predictably, preserve its state and tell the user what remains unfinished. Clear degradation is safer than an agent improvising around a missing control.
Transparency should scale with adoption
Organisations should explain the scope and limitations of the deployed capability to the people affected by it. Users need to know when automation is involved, how to challenge an outcome and where a human can intervene. That transparency improves feedback and prevents a pilot result from being interpreted as blanket approval for broader autonomy. Expansion should follow demonstrated evidence, not novelty or pressure to match a competitor announcement. The durable advantage comes from dependable integration, careful measurement and learning that survives the next model or product update.
Finally, teams should schedule a review after real usage rather than treating launch as the end of governance. The review should compare expected benefits with observed quality, cost, incidents and user behaviour, then retire workflows that do not justify their complexity. A service that works well can expand through the same evidence-based process. A service that fails should be narrowed or stopped without losing the records needed to understand why. This feedback loop is what turns experimentation into accountable operational capability.