"The costs of inference are coming down: Source? Because it sure seems like they're increasing for OpenAI, and they're effectively the entire userbase of the generative AI industry!"
Seems to me like Ed is making a very elementary mistake here. I don't think anyone has ever claimed the total amount of money spent on inference would decrease. Claims about inference cost are always about cost per quality of output (admittedly there is no consensus on how to measure the latter). If usage goes up faster than cost per unit goes down, then you spend more, but the point is that you're also getting more.
Seems to me like Ed is making a very elementary mistake here. I don't think anyone has ever claimed the total amount of money spent on inference would decrease. Claims about inference cost are always about cost per quality of output (admittedly there is no consensus on how to measure the latter). If usage goes up faster than cost per unit goes down, then you spend more, but the point is that you're also getting more.