It’s a wonderful feeling. The history of simultaneous discovery is interesting. I wonder how many other times this has happened in ML.

They had to use a search algorithm to discover this, apparently. It kind of just made sense to me. I don’t know whether there’s anything to learn from that though.

One reason I was able to do this was that I explored the Adam optimizer’s math in an intuitive way, rather than an analytical way. https://twitter.com/theshawwn/status/1623454832257548288?s=6...