(OpenRouter COO here) We are starting to test this and verify the deployments. More to come on that front -- but long story short is that we don't have good evidence that providers are doing weird stuff that materially affects model accuracy. If you have data points to the contrary, we would love them.
We are heavily incentivized to prioritize/make transparent high-quality inference and have no incentive to offer quantized/poorly-performing alternatives. We certainly hear plenty of anecdotal reports like this, but when we dig in we generally don't see it.
It does take providers time to learn how to run the models in a high quality way; my expectation is that the difference in quality will be (or already is) minimal over time. The large variance in that case was because GPT OSS had only been out for a couple of weeks.
For well-established models, our (admittedly limited) testing has not revealed much variance between providers in terms of quality. There is some but it's not like we see a couple of providers 'cheating' by secretly quantizing and clearly serving less intelligence versions of the model. We're going to get more systematic about it though and perhaps will uncover some surprises.
So what's the deal with Chutes and all the throttling and errors. Seems like users are losing their minds over this.. at least from all the reddit threads I'm seeing
Unsolicited advice: Why doesn’t open router provide hosting services for OSS models that guarantee non-quantised versions of the LLMs? Would be a win-win for everyone.
Would make very little business sense at this point - currently they have an effective monopoly on routing. Hosting would just make them one provider among a few dozen. It would make the other providers less likely to offer their services through openrouter. It would come with lots of concerns that openrouter would favor routing towards their own offerings. It would be a huge distraction to their core business which is still rapidly growing. Would need massive capital investment. And another thousand reasons I haven't thought of.
Absolutely true!!!, there will always be a 1000 wrong ideas for 1 true revolution. Sometimes these pundits would be over ambitious in their thinking or off with their timelines. What MSFT said about voice is actually coming true now with AI Voice Agents as we speak..
There is already a growing number of people who use tools like super whisper to interact with agents.
At work I'm building a "chief of staff" kind of a voice agent where you simply verbally give it tasks and it goes and gets stuff done.
Also do have a look at the latest announcements from open AI on their realtime voice
MS had great tech demos of their voice control stuff in the late 90s. Given the history I will believe this is a thing when significant numbers of people (like, hundreds of millions, not the early adopter oddball class who buy VR headsets and palmpilots and other never-quite-made-it stuff) are using voice as their primary interface with computers, and not before.
How do you think people will write code and generate images 6 months to a year from now.
Claude code, nano banana, perplexity, copilots, comet will become the default tools
I've been in tech and have been coding for 25 years now. My workflow in the past 2 months uses coding agents for 95% of my work. Just yesterday I had my agent fix and rewrite nearly 2 weeks of a 2 member in about 30 minutes. The agent's code was a lot lot better.
There there is a huge bubble, but once that bursts new patterns will emerge, Just like how ecommerce and online banking emerged, after the doc com bust
