I'm fairly certain these companies should pivot to selling / licensing AI "s/w drivers" for commodity consumer hardware that enables all these apps to run local-first or local-only.
The token cost stopped decaying as expected, as mentioned by the original 100,000k post on HN, and the move nowadays is towards more context to keep building functionality. The cost is just going to go up for inference. These companies might be better off splitting their focus between training and tooling, and canning all the capex/opex associated with inferrence.
Forget S/W engineers for a moment ... Every white collar worker I know, especially non technical folks, use ChatGPT all the time, and believe that is AI at this point. That demand isn't going to vanish overnight.
The counter argument is usually "They'll sell data", but I'm not sure you can double the number of trillion dollar data companies without some dilution of the market, and reach a billion devices / users without nation-state level infra.
It’s basically the old “what Intel giveth, Microsoft taketh away” but then with NVidia and AI shops.
Models get more computationally expensive as they start doing more things, and an equilibrium will be found what people are willing to pay per token.
I do expect the quality of output to increase incrementally, not exponentially, as models start using more compute. The real problem begins when companies like NVidia can’t make serious optimizations anymore, but history has proven that this seems unlikely.
The non S/W folks are currently all using it because it's free to a certain degree. There's no chance in hell they'll be paying for it. So the only other way for the AI companies to make money out of it is to add Ads to the whole shitshow, turning it into an even greater shitshow that not just dumbs down the planet but adds commercials on top of it.
The token cost stopped decaying as expected, as mentioned by the original 100,000k post on HN, and the move nowadays is towards more context to keep building functionality. The cost is just going to go up for inference. These companies might be better off splitting their focus between training and tooling, and canning all the capex/opex associated with inferrence.
Forget S/W engineers for a moment ... Every white collar worker I know, especially non technical folks, use ChatGPT all the time, and believe that is AI at this point. That demand isn't going to vanish overnight.
The counter argument is usually "They'll sell data", but I'm not sure you can double the number of trillion dollar data companies without some dilution of the market, and reach a billion devices / users without nation-state level infra.