the motive is probably more depressing. a normal human who just wants human interaction. people interacting with something "you" wrote just feels nice and people like that stuff.
I don't think Ed doesn't comment about the actual tech. Here are some things he has said before and please tell me if these still hold in the spirit?
> You cannot "fix" hallucinations (the times when a model authoritatively tells you something that isn't true, or creates a picture of something that isn't right), because these models are predicting things based off of tags in a dataset, which it might be able to do well but can never do so flawlessly or reliably.
ChatGPT is fairly reliable.
>Deep Research has the same problem as every other generative AI product. These models don't know anything, and thus everything they do — even "reading" and "browsing" the web — is limited by their training data and probabilistic models that can say "this is an article about a subject" and posit their relevance, but not truly understand their contents. Deep Research repeatedly citing SEO-bait as a primary source proves that these models, even when grinding their gears as hard as humanely possible, are exceedingly mediocre, deeply untrustworthy, and ultimately useless.
This is untrue in spirit.
> You can fight with me on semantics, on claiming valuations are high and how many users ChatGPT has, but look at the products and tell me any of this is really the future.
Imagine if they’d done something else.
Imagine if they’d done anything else.
Imagine if they’d have decided to unite around something other than the idea that they needed to continue growing.
Imagine, because right now that’s the closest you’re going to fucking get.
This is what he said in 2024. He really thought ChatGPT is not in the future.
There are so many examples and its clear that he's not good faith and has consistently gotten the spirit wrong.
> With the amount of talent working on this problem, you would be unwise to bet against it being solved, for any reasonable definition of solved.
I'm honestly not sure how this issue could be solved. Like, fundamentally LLMs are next (or N-forward) token predictors. They don't have any way (in and of themselves) to ground their token generations, and given that token N is dependent on all of tokens (1...n-1) then small discrepancies can easily spiral out of control.
To solve it doesn't mean we have to eliminate it completely. I think GPT has solved it to enough extent that it is reliable. You can't get it to easily hallucinate.
It depends on how much context is in the training data. I find that they make stuff up more in places where there isn't enough context (so more often in internal $work stuff).
Ed's main thesis is that cost is unsustainable for AI companies but this is clearly wrong.
The unit cost is going down and has gone down by more than 20-30x over the years. Sure, the fixed cost of training is going up but that's because of the implied returns. Once the returns to training don't happen, it would simply reduce modulo cutoff date updates. The companies have a choice to just stop training and focus on inference cost reduction.
What am I missing here? Unless the consumers decide that they are no longer willing to pay the same amount as before and their expectations are rising with prices falling, what else?
Cost, debt, difficulty forming a moat, gap between what the product promises and what it can do, and the difficulty actually raising capital required.
His style is acerbic and (imo) excessive sometimes. But he's also one of a minority of journos actually looking at the numbers and adding them up. Which seems to be a rarity
That doesn't matter if the free models are as performant in 6 months. I will never personally pay for a model I can have for free. ChatGPT 5 used to be my preferred model as a DMing help tool, now deepseek and LeChat are the one I use, and are better at what OpenAI model use to be better at. And I think the models hit their limit for my usecase, I don't need better one. I never 'reprompt' anymore, and just roll/improvise with what I got.
The only way for openAI to get my subscription back would be my country making open-weight ai or deepseek illegal. It was worth the price tbh, but they can't compete with free.
Disagree. He's cherry picking an extremely limited subset of numbers, based on a weak understanding of the industry and a lack of access to a lot of private data, and taking advantage of vulnerable people.
Well from my point of view. When they talk about gigawatt datacenters, then yes it is economically nonviable. You just need to know the scale of a gigawatt to realize that we need to start building power plants and fortifying the power grid to ship a gigawatt of power to a single location. Until the build out which takes years mind you, it is competing with other consumers of power. Lets take another huge consumer of power like a large steel mills use 100 megawatt. So if that power becomes more expensive because of datacenters, then the price of steel will go up. And if the price of steel goes up it affects a lot of things in the economy.
We are facing a situation that the short term effects are on memory and storage prices going up and lack of jet engines. Long term we wont be able to build actual buildings and ships without financing it with even more debt than today and everyone in the economy is going to service that debt through the price.
but the costs of inference have been going down 20x to 30x over the years. so how can you tell it is nonviable? unless you are saying they are not paying market rate for the inference
So, they still booked up all the ram and ssd in the world and still going to use gigawatts of power. The price of energy production is not going to go down 20x and 30x it just means that they can cram in more inference on the same energy consumption if the cost goes down. But they aren't paying the market rate for inference because everything is subsidized with debt and investors money to scale as fast as possibly. They are flushed with money and that is why they can book up all silicon production.
This claim sounds extremely fancy when AI companies bleed money, and will keep bleeding money in the foreseeable future.
I don't pretend to know the future. Maybe LLMs become economically viable and are the future, maybe not. I don't really care either way, to be frank.
And I use LLMs, btw. I pay for a ChatGPT account, but I find it only moderately useful. I always sort of question myself upon renewal date if it is worth the 20 bucks I spend monthly on it.
In no small part I keep using it to keep myself up to date on the best practices of using them in case it becomes standard.
The graph you linked seems to compare different OpenAI models in terms of "price per million tokens".
I am very skeptical of any financial information that comes from OpenAI. I have no idea how truthful those numbers are, or how creatively they can be collected to paint a rosier future for them.
Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not?
Also, I don't know this "epoch.ai" website, I don't know their stance. The website name itself does not inspire my confidence on their reporting of anything related to AI. "Eat meat, says the butcher" vibes and all.
You can claim that the AI bleeds money because training is expensive, but inference is cheap. So it will only be financially viable when they stop training models? So they would need to stop improving their capabilities entirely for it to make any sense, is that your claim?
Even if I take this claim at face value (and that would take a lot of faith I don't have to give), it doesn't sound as good as you think it does.
>To analyze the decline in LLM prices over time, we focused on the most cost-effective LLMs above a certain performance threshold at each point in time. To identify these models, we iterated through models sorted by release date. In each iteration, we added a model to the set of cheapest models if it had a lower price than all previous models that scored at or above the threshold.
Can you look at the analysis? It will make it clear. I mean its so obvious because GPT 4 costs way more than GPT 5.2-mini but much worse performance.
>Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not?
Do you think they are subsidising 900x or simply that the costs have gone down?
Overall you have shown what I feel is extreme skepticism in something that is obvious. You can literally run a model in your laptop that matches an older closed model. Costs are obviously going down, I have shown data. Use your own anecdotes and report.
Extreme skepticism in such a way doesn't do any help.
> Overall you have shown what I feel is extreme skepticism in something that is obvious.
I think you show extreme faith in something that is very obscure.
For me to believe in the analysis I would need to trust the numbers that the analysis is based upon. I see no reason why I should trust this. What sort of regulatory body or neutral third party inspects those numbers to ensure they are not a fabrication?
But you can claim I am a hater if it justifies your worldview. Skepticism is sinful for the believer.
> For our language model benchmarking, we note that we consider endpoints to be serverless when customers only pay for their usage, not a fixed rate for access to a system. Typically this means that endpoints are priced on a per token basis, often with different prices for input and output tokens.
Okay, correct me if I am wrong, so this is measuring the inference costs for clients of AI services, not the the inference costs that the AI service itself has when they offer the service?
I mean, the other guy's claim is that inference costs had come down 20x-30x. But the analysis, if I understood correctly, is based on how much clients are paying for it, not how much it actually costs.
I can charge you 20x less for a service and have massive losses for it.
It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that? In some cases the costs went down by 200x. Do you really think OpenAI is subsidising their models by 200??
Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.
For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin. What explanation is there for opensource models?
> It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that?
Possibly. I don't know.
It could be unfeasible to increase prices so much whenever a new model was released.
Any assumption made here is based on vibes. I see no reason to drop my skepticism.
> Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.
They raised an absurd amount of cash, and still bleed money to an absurd degree.
VCs make money when they exit. OpenAI only needs to "make sense" until an IPO happens. Once private investors have their exit, the markets can be left to handle the resulting dumpster fire.
> For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin.
Chinese companies are very opaque. I don't pretend to have insight into it.
Is the company behind Deepseek profitable?
> What explanation is there for opensource models?
What opensource models have to do with inference?
Your argument is that training is expensive but inference is cheap (something I see no evidence of). Why would a company give away the expensive part of the work?
>It could be unfeasible to increase prices so much whenever a new model was released.
This means you have no idea what I have been saying. A new model is costlier, but they release mini versions of old models that are way cheaper and compete with older models.
GPT 5 mini is way cheaper than GPT 4 but around the same performance
GPT-5 mini:
Input tokens: ~$0.25 per 1 M
Cached input: ~$0.025 per 1 M
Output tokens: ~$2 per 1 M
-----
GPT-4 (legacy flagship):
Input roughly $2.00 per 1 M
Output roughly $8.00 per 1 M
>Chinese companies are very opaque. I don't pretend to have insight into it.
False. The models are not opaque, you can literally download it and host it yourself. They have also released papers on how they reduced cost in certain areas.
This is literally them documenting the cost-profit ratio theoretical at 500%
>The above statistics include all user requests from web, APP, and API. If all tokens were billed at DeepSeek-R1’s pricing (*), the total daily revenue would be $562,027, with a cost profit margin of 545%.
Not only that, there are other providers hosting these opensource models, there are so many companies - just go to openrouter.com
So this is your skepticism
- openai is subsidising their models so much that each year the keep doing it 20x and eventually reached 100x reduction
- all the investors are stupid and they still invest in openai despite unprofitability
- employees of openai and anthropic who have claimed that the unit costs are not high are also lying
- all other providers are in on the lie
- the chinese models like Deepseek is also in on the lie by posting research that is not plausible
- the fact that you can run models in your laptop today that beat previous years models is also not enough
> openai is subsidising their models so much that each year the keep doing it 20x and eventually reached 100x reduction
If that's the truth, then originally they were subsidizing their models by the same factors.
This is not a great argument no matter how you cut it. And even then I would need to see evidence that this is true.
> all the investors are stupid and they still invest in openai despite unprofitability
Much to the opposite, those people are very smart. OpenAI can be extremely unprofitable and they can still profit massively through an exit event.
> employees of openai and anthropic who have claimed that the unit costs are not high are also lying
Possibly? Especially if they are in the position to profit in the case of an exit event, they would have every incentive to paint a rosier picture about the company.
> all other providers are in on the lie
I have no idea who you are talking about.
> the chinese models like Deepseek is also in on the lie by posting research that is not plausible
As I previously stated, I have no idea if Deepseek is profitable. By the looks of things, neither do you. Mentioning Deepseek's research is a non-sequitur.
> the fact that you can run models in your laptop today that beat previous years models is also not enough
He hedges so much that it's probably impossible to catch him in a contradiction or missed prediction. It must be all that practice running a PR firm for AI companies.
>You can fight with me on semantics, on claiming valuations are high and how many users ChatGPT has, but look at the products and tell me any of this is really the future.
Imagine if they’d done something else.
Imagine if they’d done anything else.
Imagine if they’d have decided to unite around something other than the idea that they needed to continue growing.
Imagine, because right now that’s the closest you’re going to fucking get.
There's only so much of it to spend before they run out.
I don't pretend to have detailed domain knowledge here, I may have seen other people's GenAI output rather than reality*, but the numbers people are throwing around for this stuff sum to trillions of USD, slightly higher than other (same caveat, perhaps also GenAI output*) claims I've seen about the total supply of money in the global venture capital markets.
* I miss the days when I could make a decent guess as to which websites were reliable and which were BS
This comment naively believes in zero sum creation of wealth.
Wealth is not taken from our consumers and given to Sam Altman. Sam and his company are creating wealth - increasing the pie.
Of course it benefits everyone while also benefiting them.
Wealth need not be redistributed to improve lives. Just the mere invention of ChatGPT and letting people purchase it and use it is enough to improve people’s lives. Redistribution does not solve any poverty problem other than transfer power.
Sam redistributing money will not sustainably change anything about prosperity or poverty.
You're talking about normal technological developments that yes generally follow Econ 101 patterns. But AGI isn't like that. If AGI or something like it comes about it won't be a normal technology. The upside case for investors is that frontier models eliminate millions of jobs and remain controlled by a small group of owners. That's why they are investing sums unprecedented in human history. If all white collar work and an increasing amount of blue collar work is supplanted by AI how do those masses of newly unemployed folks make a living without wealth redistribution?
If AI capability plateaus and ends up as a normal technological development then I agree with you that it will mostly work out for the best. But that's not the scenario I'm worried about and plenty of folks in the industry are warning that's not the most likely path at this point.
> This comment naively believes in zero sum creation of wealth.
As long as we're in a capitalist society, wealth is certainly zero-sum.
Every technological advancement that made jobs easier just allows corporations to increase their margins or increase the workload. If I automate some of my work and now only need to work 20 hours/week, I don't get 20 more hours/week of free time, I'm just given more work to do.
If someone gets completely automated out of their job, they don't get to relax and enjoy free time. They have to find a new job to pay the bills. With more and more people getting automated out of a job, UBI will become a necessity. We will need to increase taxes on corporations to fund it.
So far prices have generally gone up, which indicates the pie available is scarcer.
I am looking forward to the day where more electricity, electronics, food, and housing are produced thanks to AI; but in the mid-term it feels like an AI bubble pop would do more to bring the price back down.
While ChatGPT is a partial substitution for a college education, it doesn't satisfy the other needs I listed. I do think in the long term we'll get there, but the current situation matters.
Great, you have access to a hallucinating chat bot. The rest of us are losing access to basic computing and entertainment thanks to skyrocketing prices so that these companies can create more refined bots for you to chat with.
Yes. You can get llms to generate just about anything you want in mathematica and in particular the gpt-4.4 -> 4.5 generation had a massive improvement in mathematica code correctness in particular so it really seemed to me at that stage they specifically worked on it.
>LLMs using code to answer questions is nothing new, it's why the "how many Rs in strawberry" question doesn't trip them up anymore, because they can write a few lines of Python to answer it, run that, and return the answer.
False. It has nothing to do with tool use but just reasoning.
Oh right you're very focused on specifically the strawberry problem. I just gave that as a throwaway example. It's a solution but not necessarily the solution for something that simple.
My point was much more general, that code execution is a key part of these models ability to perform maths, analysis, and provide precise answers. It's not the only way, but a key way that's very efficient compared to more inference for CoT.
reply