I think the true edge of CoT models will come from layman usability. While I can easily prompt Claude for examples and then manually modify the code to fill in the gaps, general domain knowledge and technical understanding is absolutely required from the human sitting in front of the screen. With o1, a layman can sit in front of the computer, and ask 'I want a website for tracking deliveries for my webshop and make it pretty', and the model will do it.
So it's not so much about increased capability, but removing the expert human in the loop.
>With o1, a layman can sit in front of the computer, and ask 'I want a website for tracking deliveries for my webshop and make it pretty', and the model will do it.
I just punched that prompt into Sonnet 3.5 and o1 and I wouldn't say that o1 is doing anything better than Sonnet. o1 certainly didn't "do it", it gave me a very broad outline of how to accomplish that, from "Define requirements" to "Test and deply on Vercel"
So it's not so much about increased capability, but removing the expert human in the loop.