Streaming needed a different billing path
AWS has introduced usage-based software metering for models delivered through Amazon SageMaker AI bidirectional streaming and AWS Marketplace. The capability lets a model provider report consumption while a long-lived inference session is running, rather than charging only by the hour or trying to fit the total into a request-response header. Deepgram is the launch partner, bringing consumption-based speech-to-text and text-to-speech billing to Marketplace deployments.
Bidirectional streaming behaves differently from a conventional inference request. A client can send audio and receive results over an ongoing connection that lasts seconds or minutes, while usage accumulates throughout the session. The final quantity is not known when the connection begins. AWS therefore needed a mechanism that could report incremental units without disrupting the data stream carrying the actual transcription or generated speech.
The design adds a second WebSocket beside the inference connection. The data connection continues to carry model input and output. A companion metadata connection sends small JSON metering records, with the same SageMaker request identifier linking the two channels. Providers opt in through a response header and can report units as they accrue, reducing the amount of consumption left unreported if a session ends unexpectedly.
A companion channel carries metering records
The provider defines what a unit represents and maps it to the Marketplace offer. For a speech model, that might be a second of processed audio. Twelve minutes of billable audio would then be reported as 720 units, but an open connection lasting twelve minutes does not automatically establish the billable quantity. Buyers need to understand the provider’s counting rules rather than assume connection time and processed media are identical.
SageMaker endpoint infrastructure remains a separate charge while the endpoint is provisioned. The new mechanism covers the Marketplace software component, not the entire cost of running the service. This separation is important for cost comparisons. A low per-unit model rate can still sit on expensive or under-utilised infrastructure, while a higher software price may be economical if the endpoint is shared efficiently and idle capacity is controlled.
AWS allows providers to choose whether inference stops when the metadata stream fails. If metering is marked required, losing the companion connection terminates the data session. If it is optional, inference continues while metering stops. That is a real business trade-off between service availability and billing continuity. Providers should decide deliberately and tell customers how a failure will appear rather than leave support teams to infer it during an incident.
Availability and billing can fail differently
Transport delivery alone does not prove that a unit was billed. AWS recommends idempotent client tokens and reconciliation against Marketplace reports. Providers need monitoring for rejected, malformed or duplicated records and a process for customer disputes. Customers should compare service logs, processed media and invoice data, especially during the first months of adoption. A technically successful stream can still produce incomplete commercial records.
For model vendors, the feature creates a path to sell live AI services through consolidated AWS billing. Private offers can carry negotiated per-unit or fixed terms, and customers avoid a separate billing relationship with each provider. The onboarding work is not trivial: vendors must implement the metadata endpoint, define units, test failure behaviour and production-harden the example integration before listing a paid product.
For buyers, usage pricing can align cost more closely with value than an hourly software licence. Contact centres, captioning services and voice agents may have bursty demand where paying for processed audio is easier to allocate. The organisation still needs budget alerts and session limits. An agent that loops, leaves a microphone active or repeatedly reconnects can turn a granular rate into an unexpectedly large bill.
Buyers need a precise unit definition
Security review should cover both connections. The metadata channel carries billing information rather than model content, but it still participates in an authenticated session and can affect whether inference remains available. Teams should test request correlation, replay handling, rate limits and behaviour when one channel closes before the other. Logs must avoid exposing customer audio or credentials while retaining enough evidence to investigate discrepancies.
Deepgram’s launch implementation gives AWS a concrete production-oriented use case. It also sets expectations that other real-time model vendors can follow. The broader significance is that marketplaces are adapting from static software subscriptions to AI services whose consumption changes moment by moment. Clear metering semantics become part of the product contract, not a detail hidden behind the inference API.
Usage-based streaming pricing is most useful when it is transparent, reconcilable and paired with infrastructure cost controls. AWS has supplied the mechanism for reporting units during a live session. Providers must define and operate it honestly, while customers must measure the full workload cost. If both sides do that work, Marketplace can support real-time AI purchasing without forcing every vendor to invent a separate commercial stack.