Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Exactly. The idea that massive compute would lead to massive improvement was non-obvious.

It was quite reasonable to think that there would be rapidly diminishing returns in model size.

Wrong, in hindsight, but that's how hindsight is.



>The idea that massive compute would lead to massive improvement was non-obvious.

Honestly no, it was obvious, but only if you listened to those pie in the sky singularity people. It was quite common for them to say, add lots of nodes and transistors and a bunch of layers and stir in some math and intelligence will pop out.

The groups talking about minimal data and processing have not had any breakthroughs in, like forever.


Google and all the big players in AI have known they need tons of data and hence compute power for processing it, for a very long time, way before OpenAI even existed. Anyone getting involved in that game would have definitely known.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: