I still haven't found "that one visualisation" that makes the attention concept in Transformers as easily understood as these CNNs.

If someone here on HN has a link to a page that has helped them get to the Eureka-point of fully grasping attention layers, feel free to share!

I found this video helpful for understanding transformers in general, but it covers attention too: https://www.youtube.com/watch?v=kWLed8o5M2Y

The short version (as I understand it) is that you use a neural network to weight pairs of inputs by their importance to each other. That lets you get rid of unimportant information while keeping what actually is important.

Hi! I'm the creator of the site. Good news: I'm currently working on animations and an explainer video on transformers and self-attention. The best way to be notified is probably to subscribe to my YouTube channel and hit the bell icon for notifications.

You mean you would be excited to see attention animations? The page presents convolutions not attention.