NVIDIA TensorRT Edge-LLM brings Nemotron inference from DGX Spark to Thor platforms
NVIDIA has published a TensorRT Edge-LLM workflow for running Nemotron-3.5 Lightning on a 64GB DGX Spark and serving it through an OpenAI-compatible local API. The same lightweight C++ runtime targets Jetson Thor, DRIVE AGX Thor and IGX Thor, while optional DSpark speculative decoding materially increases measured generation speed.