Sorry for the hassle Sam. We're seeing decreasing error rates on the API now. This was painful -- appreciate your patience while we work through the issue with one of our infra providers.
I bet your day was more filled with hassle! But genuinely appreciate the response. One thing it’s really highlighted to me is that you’re miles ahead of the alternative APIs available. It’s also highlighted quite how much I’ve come to rely on not just the API but other little workflows I’ve made in Claude itself. I felt quite bereft for most of the morning.
Someone who has built infrastructure or system prompts that use Opus will probably continue with Opus until they verify that everything works on Sonnet 3.5
Benchmarks don't cover all possible use cases, for one. There's always the possibility that a model does better on every benchmark thrown at it, but for your specific use case it does worse in practice.
