News
Updates, product notes and practical coverage for AI platforms and operators.
AWS Marketplace now supports usage-based software metering for bidirectional streaming inference on Amazon SageMaker AI. Model providers can report consumption during live sessions through a companion metadata WebSocket, while customers retain separate endpoint infrastructure charges. Deepgram is the launch partner for speech-to-text and text-to-speech models.
AWS has published a measurement-driven autoscaling method for Kokoro-82M text-to-speech on SageMaker AI. It derives thresholds from per-instance concurrency before an SLO breach and the full time required to add capacity. The example measures 320.2 seconds of planned scale-out time and uses gated CDK deployment, drift checks, admission control and asymmetric cooldowns.
AWS has published a technical guide to SageMaker AI training session APIs, which expose sampling, forward-backward, optimisation and checkpoint operations against a live GPU-backed session. Developers can implement supervised fine-tuning, preference optimisation or reinforcement learning locally while SageMaker handles distributed execution, model loading, fault recovery, isolation and managed accelerator capacity.
AWS has published an end-to-end SageMaker AI multi-turn reinforcement learning guide using a Wordle agent hosted on Bedrock AgentCore. The example trains GPT-OSS-20B from episode-level rewards, lifting held-out solve rates from 61 to 84 per cent at serving settings while documenting reward design, validation differences, token costs and deployment requirements.
AWS has published a reusable CDK construct that deregisters SageMaker endpoint autoscaling before configuration updates, waits for the endpoint to return to service, then restores the scalable target and policy. The pattern addresses failed CloudFormation rollbacks and silent loss of scaling, using custom resources, dependency ordering, bounded waiters and configuration hashes.
AWS has published a SageMaker AI pattern that streams each completed agent sentence to the Kokoro-82M text-to-speech model instead of waiting for a full answer. Tests reduced transport time to first audio by 52.4 per cent and delivered 2.4-to-3.5-times faster end-to-end starts, while documenting binary transport, buffering and dependency-security trade-offs.
AWS has made SageMaker AI training sessions generally available for custom post-training loops without managing GPU clusters. Developers can implement supervised fine-tuning, preference optimisation or multi-turn reinforcement learning while SageMaker handles allocation, distributed execution, recovery and checkpoints. Initial model support includes GPT-OSS, Gemma and Qwen families with token-based metering.