It seems deeply overvalued to me, as it's pretty likely that NVIDIA's competitors will have parallel computation hardware that competes well enough. I'm personally not a fan of NVIDIA's drivers or the reliability of their hardware.
> I'm personally not a fan of NVIDIA's drivers or the reliability of their hardware.
As compared to AMD or Intel? I wish there was real competition to Nvidia but there isn't. I'm not a fan of their defacto monopoly but they do have the best product on the market and their competition has been asleep for 10 years. AMD and Intel barely knew what deep learning was 10 years ago (and certainly did not appreciate the opportunity) and Nvidia was already investing heavily.
Intel wasted the last decade because they elected a shareholder optimizing CEO who chose to prioritize shareholder payouts and attempts at controllig the ecosystem rather than pushing it forward over investments.
Now they're completely outclassed by TSMC and have to partner with UMC to compete.
The point is that thing can change. Intel was king a decade ago. TSMC is on the summit at the moment. Intel can be back in a few years if their plan works out:
As compared to a hypothetical product that is better than whats available today. With literally a trillion dollars on the line I find it very difficult to believe no one will come and scoop this opportunity up. The real value in GPUs is the datacenter segment which largely didnt exist, certainly not in that state it is today before the LLM take off. Takes time to develop products but theyll arrive eventually.
Well there's not really a trillion dollars on the line. This is a valuation, not revenue.
All this says is that AI and LLMs are extremely over hyped, and the market believes Nvidia's tech is the only viable supplier of the platform LLMs run on.
These are things we already knew, so it's not surprising the market is quadrupling what it thinks Nvidia is worth.
Yes there is a trillion dollars on the line here. Market cap is what matters to investors end of the day, everything else is just influencing it. Yes the market is saying Nvdia is the only supplier for AI and LLM hype, but its also saying they will continue to be the only supplier long term and that seems deeply flawed to me.
While there's certainly a feedback cycle involved, market cap is decided by investors in the large. The stock price is just reflecting buys and sells of those investors. People believe AI has a lot of money and hype in it, and Nvidia supplies necessary equipment for AI, so they buy Nvidia. As a result the stock price goes up, and the market capitalization goes up accordingly.
As a thought experiment, imagine buying 50% of a company on the open market and then seeing the price go up accordingly as the market does, then saying it must be even more valuable than I thought! And buying the other half of the company at the higher price. You caused that "value" by buying.
No one investor has a trillion dollar opportunity, and no competitor does either. Making an assumption that a competitor will come along and zero out Nvidia because they have a better AI chip is not rational. For one the value of Nvidia isn't solely based on this, and by the time you've made your AI chip the market is going to have changed.
Considering how insanely inflated the public's expectations of what LLMs can do, despite the fact that they are only a mirage of intelligence, it would probably be foolhardy to build a new AI chip to replace them.
I suppose for AMD they could see their stock increased by such a margin if they could just produce a chip that convinced the market, but were they not already trying to do that?
Yep, I remember 8 years ago Intel trying to sell us their CPU solutions for AI. It was hard to make things work and in the end performance were not there. We were comparing against consumer gpu like the GTX 1080.
Before AI/ML was hot, and before even the Bitcoin paper was released. NVidia was investigating/experimenting/investing in the concept before there was any kind of 'killer app' for it.
And even better, NVidia understood not everyone wants to use plain old C for their GPGPU coding, and early on staring with CUDA 3.0 in 2010, introduced C++ support and PTX.
Later on they acquired PGI, which thanks to PTX, had C, C++ and Fortran compilers, thus adding Fortran into the mix.
Followed along by all the IDE, graphical debuggers tooling and library ecosystem.
Meanwhile Intel and AMD were doing who knows what at Khronos stuck in their "C is good enough" mentality, and barely released useful developer experiences.
My statements based on high level meetings I had at the time with all 3 companies when I had started an early neural network PaaS company and was looking for them to invest. Nvidia knew what they were talking about and were already moving that direction, Intel heard about deep learning somewhere but didn't believe there was anything there, and AMD didn't know anything about anything.
Seems like a tale already told on "Only The Paranoid Survive" by Andrew Grove. Now Jensen should add some chapters. BTW I just discovered that their web page/brand is fully invested in AI, the title is: "World Leader in Artificial Intelligence Computing".
I’m not sure I would bet an ads/spyware company with admittedly deep deep pockets would beat a specialized hyper focused company at its core business, especially as they have deeper pockets now with 30 years of experience in the sub sector. If Nvidia were wandering in the desert like Cisco I could believe it. But Nvidia isn’t, and I don’t believe Google, Amazon, or others will beat Nvidia at their own game. (Given my time in FAANG, I also speak from inside knowledge of how deeply f’ed these companies are - making your own arm chip or network adapter isn’t the same thing as taking on high end designers at their own game)
It's interesting that you would name drop Cisco. They exploded in market valuation in 1999/2000, having a near-monopoly on infrastructure that was in high demand due to the Web boom. Nvidia similarly profited from the Crypto/AI boom, but I wonder whether that is bound to end similarly.
Have you worked at a large (say 500+ employee) company that you would say isn't "deeply f'ed"?
I once read a fascinating corporate history of Xerox, and that company became deeply, deeply f'ed up in ways that are of their time but do have strong parallels to the issues I understand that FAANG, particularly Google, have.
> Have you worked at a large (say 500+ employee) company that you would say isn't "deeply f'ed"?
You get a weird insider bias where all you see are bug reports and problems, so you get the impression the product is shit, even if 99% of customers love it.
One of them was called Xerox: American Samurai, and dates from 1986 (I read it when I was a teenager in the early 1990s, I think). There was another one from the early 1990s, I believe.
This book lauded Xerox's success at reforming its corporate culture, regaining a strong position in the photocopier market and even spent a chapter detailing their success in getting into electronic typewriters!
With the benefit of hindsight the company didn't survive its core product losing all relevance any better than Kodak did, but that was still some way in the future (if foreseeable by the 1980s).
That said, much of the material about a hugely bloated organisation, with a sclerotic bureaucracy and lots of cushy middle managers assembled through a previous period of explosive growth, turning out poor-quality product, sounds very reminiscent of some of what we now hear about the current big tech companies.
The title reflects another American obsession at the time - the idea that the US was "losing" to Japan.
>Given my time in FAANG, I also speak from inside knowledge of how deeply f’ed these companies are
Please tell us more of your expertise and deep insight, FAANG employee #1,908,680.
Google has developed their own chips. Apple has developed their own chips. It’s really not that hard if your pockets are deep enough or the bottom line checks out.
When Apple launches their AI offering this year, it’s not going to need NVIDIA.
They have, over many years and not against a company at the top of their game and with considerable help. Apples move is an ARM chip, not an Apple chip. Googles chips are competitive in spaces like network infrastructure. TPUs are illustrative of their inability to provide a viable alternative.
Apple isn’t going to launch with an Nvidia killing alternative, I’ll bet you $1,908,680 it’s backed by Nvidia.
They will likely use their neutral chips for local models, but their data center stuff will be 100% Nvidia.
The interesting question isn't whose hardware they use for the launch, it's what the public-facing software API looks like. Apple isn't likely to directly expose CUDA. At which point they're free to swap out the hardware with whatever they want at any time.
Also, Apple has a longstanding dislike for Nvidia and even if they weren't going to design their own chips at launch, they could be using AMD.
AMD has taken more CPU market share from Intel than Apple.
But the weird thing about the "Nvidia will remain undefeated forever" theory is that it seems to assume they have some kind of permanent advantage.
Nvidia was well positioned to make an early investment in this because they had the right combination of existing technology and resources. Other companies would have had to invest more because they were starting from a different place (e.g. Microsoft), or didn't have the resources to invest at the time (AMD).
But now the market is proven and there are more than half a dozen 800 pound gorillas who all want a piece of it. It's like betting that Tesla will retain their 2022 market share in electric car market even as the market grows and everyone else gets in. Maybe some of the others will stumble, but all of them? Apple, AMD, Google, Intel, Microsoft, Amazon and Facebook?
And even a Tesla optimist would presumably admit that maintaining a third of the EV market would be a huge win for Tesla, as the EV market becomes "the car market" going forward. Maybe that won't happen, but it's at least within the realm of possibility -- maybe some of the existing carmakers stick the transition and some of them fail, but Tesla remains the biggest one, that's not impossible.
But maintaining the 80% they had a few years ago, much less 100%? That's not optimism, it's fantasizing.
Intel knows how to make software and libraries for their hardware, which is the thing people keep lamenting about AMD. Intel's current GPUs are mediocre but priced competitively for what they are, and Intel having more competitive hardware in the future is not implausible.
Which could lead to Intel realizing the opportunity they have. Create decent libraries that work across every vendor's GPUs. In the short term this helps AMD at the expense of Nvidia, which in itself helps Intel by preventing Nvidia from maintaining a moat. In the medium term Intel then has people using Intel's libraries to write code that will work on their future GPUs and then their problem is limited to producing competitive hardware.
Their biggest competitor in the GPU space is AMD who have spent years chasing deadlock issues that won't stay fixed, playing whackamole, in between trying to do actual driver development. Following the amdgpu mailing list is not fun.
A bit funny thing is that they are essentially software company which focuses on software quality, if you look at the stats how many people are working with software over there. And still it is not good enough?
I'd welcome a lot fewer software developers, so I get drivers rather than the bloated driver lifestyle synergy experience with cloud mess. So many companies spend so much money developing software that alienates and aggravates their customers. I blame HP, who by my recollection started it with printer drivers that required a custom installer and a CD.
Not having done any GPU programming...is most of the code out there tied to the Nvidia architecture? I mean, if AMD or somebody else builds a better mousetrap, how much work would it be to switch?
Most of the AI and ML related programming uses frameworks that abstract away what brand of GPU you are using. They could care less whether it is nvidia or AMD.
This is true for making things just work, however to really squeeze performance out of a GPU, you need to go lower level and that is tied to the architecture.
This has happened before and it will probably go the same way. Software and compilers will make up the difference, or hardware will become so cheap and ubiquitous it wont super matter.
In 3-5 years what will a 10% performance difference matter to you? Then calculate how much that 10% performance difference is going to cost in real dollars to run on nvidia hw and then the fun math should start.
Performance for GPUs isn't just speed, but also power efficiency. The complexity of GPUs doesn't lend itself to just being solved with better tooling. They are also not going to get cheaper... especially the high end ones with tons of HBM3 memory.
Given that data centers only have so much power and AI really needs to be in the same data center as the data, if you can squeeze out a bit more power efficiency so you can fit more cards, you are getting gains there as well.
When I was mining ethereum, the guy who wrote the mining software used an oscilloscope to squeeze an an extra 5-10% out of our cards and that was after having used them for years. That translated to saving about 1 MW of power across all of our data centers.
Let me also remind you that GPUs are silicon snowflakes. No two perform exactly the same. They all require very specific individual tuning to get the best performance out of them. This tuning is not even at the software level, but actual changes to voltage/memory timings/clock speeds.
You are right to worry about power efficiency. Though do keep in mind that power is also fungible with money, especially in a data centre.
I suspect a lot of AI inference (thought probably not the majority) will happen on mobile devices in the future. There power is also at a premium, and less fungible with money.
> Though do keep in mind that power is also fungible with money, especially in a data centre.
Untrue. I have filled 3 very large data centers where there was no more power to be had. Data centers are constrained by power. At some limit, you can't just spend more money to get more power.
It also becomes a cooling issue, the more power your GPUs consume, the more heat they generate, the more cooling that is required, the more power that is required for cooling. Often measured in PUE.
Hate to break it to you. That is getting harder and harder. Certainly you can get a couple mw, but if you want 50, you are going to find that to be extremely challenging, if not impossible.
The large ones being built today are spoken for already. That is how crazy the demand is right now. People aren't talking about it in the general news. They can't build them fast enough cause things like transformers, generators and cooling are all having supply issues.
Even still, power is limited. You can build DC's but if you can't power them... what are you going to do? This isn't just throw more money at the problem.
Have you noticed that data center stock, like EQIX, are at all time highs?
> Have you noticed that data center stock, like EQIX, are at all time highs?
Though the FTSE All-World index (or the S&P 500) is also at all time highs, so I would expect most stocks to be at all time highs, too.
> Even still, power is limited. You can build DC's but if you can't power them... what are you going to do? This isn't just throw more money at the problem.
I guess you can try to outbid other people? But thanks: I didn't know the data-centre-building industry was so supply constrained at the moment.
I knew you'd say that. SMCI though. It isn't just the macro. It is this AI stuff and has been going on quietly behind the scenes for the last 1.5-2 years now. Unless you're deep in the business, it just doesn't make the news headlines because it is all so intrinsically hush hush.
> I didn't know the data-centre-building industry was so supply constrained at the moment.
The whole supply chain is borked. Try to buy 800G mellanox networking gear. 52 week lead time. I've got a fairly special $250 cable I need that I can't get until April. I could go on and on...
I've seen some of that playing out in a business that was using GPUs for deep learning as applied to financial market making. They were throwing a lot of money at nvidia, too.
I wonder if it's enough money in total in AI to show up in countrywide GDP figures anytime soon. Because either AI's hunger for ever more computing power has to slow down, or world GDP has to increase markedly.
Well, I'm talking about the rate of increasing slowing.
Given the speed of light as an upper limit, in the very long run we can at most have a cubic growth, not an exponential growth. Something will have to give eventually.
(OK, you also probably need to Bekenstein bound. Otherwise, you could try sticking more and more into information into the same amount of space. But there's a limit to that, before things turn into black holes.)
We are so, so, so far away from compilers that could automatically help you, say, rewrite an operation to achieve high warp occupancy. These are not trivial performance optimizations - sometimes the algorithm itself fundamentally changes when you target the CUDA runtime, because of complexities in the scheduler and memory subsystems.
I think there is no way that you will see compilers that advanced within 3 years, sadly.
Actually, this hasn't been shown yet. AMD showed some "on paper" specs which could "theoretically" be faster than a H100, but they didn't show any practical tests which could be recreated by third parties. Also, some of the tests AMD ran on H100's were deliberately not using the correctly optimized software, massively slowing down the H100 performance...
Google Pixels have a "neural core", and they have edge TPUs in addition to their data center TPUs. However Google hardware seems much less poised to take immediate advantage of the AI gold rush.
The last decade of Google datacenter engineering can be viewed as an elaborate plan to avoid paying either Intel or Nvidia any more money than was really necessary.
Nobody other than Anthropic and a few other large cloud customers like that really cares what Google is working on. They bet the farm on a DL framework (Jax) with 2% market share. That’s a very deep hole to climb out of, particularly considering the popularity of PyTorch in generative AI space.
