Baseten has made Thinking Machines Lab’s new Inkling model available through its Model APIs and Dedicated Inference service on the model’s first day of release. The move gives developers and enterprises a managed way to test and deploy a very large open-weight model without first assembling the substantial infrastructure the model requires.
According to Baseten, Inkling is a general-purpose multimodal model that accepts text, images and audio and produces text. Thinking Machines Lab describes it as the first model in a new family intended to be adaptable through fine-tuning. Baseten’s announcement is therefore both a model-availability update and a test of whether managed inference platforms can make unusually large open models practical for more teams.
What Baseten is offering
Inkling is available through two Baseten routes. Model APIs provide a quicker way to experiment with the model behind a hosted endpoint, while Dedicated Inference is intended for organisations that need a deployment with more control over capacity and operational settings. Baseten says it is continuing to optimise performance, so early users should treat current throughput and latency as an initial baseline rather than a finished result.
The model uses a sparse mixture-of-experts architecture with 975 billion total parameters and 41 billion active parameters. Baseten says each token is routed through six of 256 specialised experts plus two shared experts. The model supports a context window of up to one million tokens and was trained across text, image, audio and video data, although its output is text.
Those specifications create a demanding serving problem. Baseten cites a minimum aggregate GPU-memory requirement of about 2 TB for the BF16 checkpoint and at least 600 GB for the NVFP4 checkpoint. Open weights provide deployment and customisation flexibility, but they do not remove the cost, engineering or capacity constraints associated with operating a model at this scale.
How the managed deployment works
Baseten says its inference stack combines autoscaling, distributed weight delivery and capacity across more than 20 cloud providers. Its autoscaler adds or removes replicas as demand changes, while the Baseten Delivery Network keeps model weights close to available compute. The company claims this delivery system can produce cold starts two to three times faster, although prospective users should validate that claim against their own traffic patterns and regions.
Multi-cloud placement is another part of the proposition. Baseten says workloads can be placed with another provider if suitable GPU capacity is constrained in one cloud. That may reduce some supply risk, but organisations will still need to examine data-location requirements, service-level commitments, observability, security controls and the consistency of performance across regions.
What developers should assess
Inkling is positioned for agentic systems, coding assistants, chatbots and retrieval-augmented generation. Its multimodal inputs and large context window broaden the potential use cases, but the announcement does not establish which workloads will deliver a better cost-to-quality result than smaller or more specialised models.
Teams evaluating the hosted service should measure end-to-end latency, token costs, concurrency, tool-use reliability and quality on domain-specific tasks. They should also check how image and audio inputs affect processing time, whether fine-tuned versions can be moved between environments, and which controls are available for retention and access to submitted data.
Open-weight availability also changes the procurement comparison. A team can weigh Baseten’s managed endpoint against self-hosting, another inference provider or a smaller closed model. That comparison should include engineering labour, idle accelerator capacity, network transfer, recovery time and the cost of maintaining optimised runtimes, not just the advertised per-token rate.
Customisation will require separate scrutiny. Thinking Machines Lab is making Inkling’s weights available and supports fine-tuning through its Tinker service, but Baseten’s announcement does not set out every portability, training or licensing condition for a customised deployment. Buyers should review the model card and licence, document the training data they add, and confirm how resulting weights and logs can be exported.
Benchmark results cited by the providers span reasoning, coding, vision, audio and safety, but scores from different tests are not directly comparable. An evaluation should use representative prompts, tools and documents, include adversarial and safety cases, and compare output quality at matched latency and cost. A one-million-token context window is useful only if retrieval and reasoning remain dependable across the full input.
The day-one release is significant because it shortens the path from an open-weight model announcement to a callable production service. It does not, by itself, prove that Inkling is economical or reliable for sustained workloads. Baseten’s ongoing optimisation work and the model’s infrastructure footprint make controlled pilots the sensible next step.