There has been growing curiosity around what Mira Murati’s Thinking Machines Lab is building with its $2 billion seed funding and a team of former OpenAI researchers. On Wednesday, the lab gave the public its first glimpse into its research efforts through a blog post exploring one of AI’s fundamental challenges — the unpredictability of large language model (LLM) responses.
The post, titled “Defeating Nondeterminism in LLM Inference,” seeks to break down why AI models often deliver different answers when asked the same question repeatedly. In today’s AI landscape, this variability has been largely accepted as a given, with LLMs considered inherently non-deterministic. But Thinking Machines Lab sees this as a problem worth solving — and potentially one that can be solved.
The post’s author, researcher Horace He, argues that much of the randomness in model output stems from the way GPU kernels — the small programs that run within Nvidia chips — are pieced together during inference (the process that happens after a user submits a prompt). By carefully controlling how this orchestration is handled, He suggests it may be possible to make model outputs more deterministic and reproducible.
This breakthrough could have significant implications. Beyond improving reliability for enterprise use cases and scientific research, reproducible responses could enhance reinforcement learning (RL) — the training method where AI models are rewarded for correct answers. As He notes, when responses vary too widely, RL data becomes noisy and less effective. Making outputs consistent could make RL training “smoother” and improve the customization of AI models for businesses, which the lab has reportedly told investors is part of its strategy.
Murati, OpenAI’s former CTO, has said the lab’s first product will be unveiled in the coming months and will be “useful for researchers and startups developing custom models.” Whether this product incorporates the reproducibility work remains to be seen.
Thinking Machines Lab also announced that it will regularly publish blogs, code, and research under its new series “Connectionism” to foster open collaboration. The first post signals that Murati’s lab is taking on some of the toughest challenges in AI research. The real question now is whether it can turn these insights into products that justify its $12 billion valuation.
