There's a lot of push for inference hardware now (e.g. Ironwood TPUs). How does Nvidia maintain an edge there?
Also, I think the market has to expand beyond LLMs to areas like robotics and self driving cars (and they need to have real success) for Nvidia to maintain this valuation. I don't think only LLMs are enough because I don't see code assist/image generation/chatbots as a massive market.
That's because Nvidia is offering a full ecosystem stack with HW, SW and networking clusters.
And that gives customers the most flexibility. Nvidia dominates training and is highly competitive in inferencing. At the same time, SW improvements speed up single node and networking performance. H100 released 3 years ago is today several times faster than it was on release with constant SW updates.
Customers who buy Nvidia for training today can use the older GPUs from Nvidia for inferencing later. And Nvidia supports even V100 still in SW updates and speed improvements. And since all is based on the same SW ecosystem, it allows for more seamless operations for customers. You can mix different Nvidia GPU clusters but you can't easily mix Nvidia solutions with other solutions.
That is also why Nvidia has always been dominant and that's flexibility. NVFP4 is a good example of what they do to stay ahead. And it is even supported by Hopper so any old customer can use Nvidia's new format to further improve model training performance. Suddenly old Hopper clusters become more valuable with some SW releases by Nvidia.
Nvidia has a track record which no competitor can match. Going with Nvidia is no mistake today while going with any competitor is a risky bet. If you spend billions, you think twice about making bets.
Also, I think the market has to expand beyond LLMs to areas like robotics and self driving cars (and they need to have real success) for Nvidia to maintain this valuation. I don't think only LLMs are enough because I don't see code assist/image generation/chatbots as a massive market.