I’m not going to be defaulting to other providers for new tasks - just putting a fail-over in place.
Out of interest what small set of tasks do you find Claude to be best for? Because I find it to be significantly better for most things. The only thing I have found it not better at for my use cases is identifying specific pieces of (somewhat specialist) machinery and equipment from images, where I’m still getting stronger results via OpenAI.
We mostly do multimodal tasks (vision + text), and there the differences between flagship models are still much bigger. For us, the benchmarks showing all of them being close are pretty meaningless, it really depends on the task when vision is involved.
Our pure text tasks are generally quite simple, so for price+speed reasons those don't use Sonnet but instead Llama 3.0, very-old-version 3.5 Turbo (newer versions are awful) or 4o-mini.
Out of interest what small set of tasks do you find Claude to be best for? Because I find it to be significantly better for most things. The only thing I have found it not better at for my use cases is identifying specific pieces of (somewhat specialist) machinery and equipment from images, where I’m still getting stronger results via OpenAI.