New benchmark for competitive coding dropped yesterday - https://livecodebenchpro.com/

Apparently models are not doing great for problems out of distribution.

It goes to show that the LLMs aren't intelligent in the way humans are. LLMs are a really great replacement for googling though

It would only be interesting if the problem was truly novel. If the AI has already been trained on the problem it’ll just push out a solution.

It already happened. Last year AI submissions completely destroyed AoC, as far as I remember.