Takeaways from GTC Berlin 2026

A short post to share a few takeaways from NVIDIA GTC Berlin 2026. My interests are in inference and infrastructure, so most points concern those.

Everyone runs every engine: Almost all the neoclouds (CoreWeave, Lambda, Nebius, Crusoe, Verda, Radiant etc.) and inference providers (Fireworks, Baseten, Together AI, DeepInfra etc.) are running all three open source inference engines (SGLang, vLLM, TensorRT-LLM) in production. Not every model performs best in the same engine, and companies would rather take on additional complexity and get closer to peak performance for each model than simplify the stack but leave performance on the table.
Most companies in the inference space are not afraid to get their hands dirty: Many teams said that a substantial portion of their serving stack was developed in-house, whether it was written from scratch or forked from an open source project. This actually makes sense, given that the moat for inference providers is in their serving efficiency. Companies like Together AI and Fireworks in particular have very strong internal R&D orgs; Tri Dao of FlashAttention fame is Chief Scientist at Together AI, and Fireworks was founded by a group largely consisting of the most senior members of Meta’s PyTorch team. This is all to say that the companies most focused on inference have the technical chops to do things themselves. Naturally, everyone was pretty tight-lipped about the details of these internally developed components.
Neoclouds and inference providers are converging, and both are moving up the stack: Neoclouds, who mostly own their infrastructure, build their own datacentres, and earn revenue mainly through cluster rental and long-term commitments, have launched inference and post-training products. Inference providers are (trying to) move the other way and are beginning to offer products that bring customers closer to the metal, as well as making efforts to own at least a portion of their compute pool. Part of the reason for this is that the serverless inference product (e.g. a Kimi K3 API endpoint) is a particularly brutal category for providers margin-wise, because OpenRouter and the like make switching costs so low. Both camps therefore have an incentive to offer more sophisticated products — like managed training, fine-tuning, and reinforcement learning — which are more involved but promise higher margins and stickier customers. Most of these companies have had a managed fine-tuning product for some time; RL is the latest addition for many of them.
Dynamo gets mentioned a lot: It’s arguably the most widely adopted component of the inference stack, at least amongst the teams that I spoke to. It seems like something worth getting familiar with, even contributing to, for those wishing to get into the inference space.
NVIDIA’s marketing / sales people are very technical: They’re also very nice!
Prefill-decode disaggregation gets talked about the most, but there are other things we can disaggregate too: For instance, when employing speculative decoding, we can disaggregate by putting the small draft model on a dedicated device. This kind of disaggregation can be as important for performance as PD disaggregation, or even more so, but it’s not as universal a win.
NVIDIA’s AI-RAN concept is fascinating: Basically, cell sites nowadays contain specialised ASICs for the signal processing that turns radio waves into data. But GPUs are capable of doing this work too, and after a combination of hardware and software developments, NVIDIA says its Aerial RAN Computer (ARC) platform can now match the performance of the ASICs designed for that role, albeit at a higher power draw. The motivation for using a GPU instead of an ASIC is edge inference. Telcos place their cell sites in a way that brings as much of the population as possible within as close a range as possible. Edge inference spreads small clusters of GPUs out across a region, putting them closer to the customer and thus minimising latency, instead of all inference traffic going to one big inference cluster in a single location. So cell sites are ideal places to serve inference from, and if the GPUs are there anyway for the radio processing, their surplus compute can run inference workloads too, giving telcos an extra revenue stream. NVIDIA calls this approach ‘AI-RAN’. There are also other fun things you can do with such a platform, like using ML to improve spectral efficiency, or running inference for voice models on the same machine the call is being carried over. NVIDIA’s next-generation platform, ARC-Pro, which Nokia is building its AI-RAN products on, is expected to be generally available in the first half of 2027.
Large coding agent bills are one of the biggest drivers of open model adoption: A large proportion of serverless inference usage is coming from companies who have run up sky-high Claude Code bills, and are now looking to migrate their engineers onto open source models. This surprised me, as I had assumed that much of the serverless inference revenue was coming from use of those models in customer-facing products, as opposed to internal usage for development. Another interesting thing is that many inference providers have found that their customers’ engineers are unusually sticky to the Claude Code harness, even if they’re forced to use an open source model underneath. Hence, inference providers have invested extra effort into making the open source models they serve compatible with the proprietary coding agents / harnesses people are already loyal to, even though this adds extra headaches.