This is great. I wonder if the price decrease comes from the competition (on the side of anthropic and from local LLMs). If so, I guess we will have to wait for a general GPT-4 competitor to come along before we see price decreases there aswell. Right now it's quiet expensive. We are incorporating it in a new product in the education space and we have to be fairly conservative in rate limiting things so that the cost won't go out of hand.
I also wonder how much of an impact the new Nvidia HGX systems will have on the medium term infra cost on running these services and whether we will see some benefits from that.
Right this is a very exciting release but disappointing that there was no price reduction at all or rate limit increase for gpt-4. I guess it just uses a lot of GPU and RAM.
> Right this is a very exciting release but disappointing that there was no price reduction at all or rate limit increase for gpt-4
They are planning on reducing the pricing from $infinite to $current-listed (or, viewed another way, to increase the quota from 0 to current-listed) by clearing the waiting list.
This, obviously, doesn’t benefit (may even, competitively, hurt) those who already have GPT-4 access, but for everyone else, its a win.
I also wonder how much of an impact the new Nvidia HGX systems will have on the medium term infra cost on running these services and whether we will see some benefits from that.