If you just want to understand the Transformer, here is a clean implementation: ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		abakus on Aug 17, 2020 \| parent \| context \| favorite \| on: Karpathy's MinGPT If you just want to understand the Transformer, here is a clean implementation: https://github.com/blue-season/pywarm/blob/master/examples/t...

chronolitus on Aug 18, 2020 [–]

and here's a breakdown of the architecture:

http://dugas.ch/artificial_curiosity/GPT_architecture.html

odnes on Aug 18, 2020 | [–]

These 4 videos (~45 mins) do an excellent job at explaining attention, multi-headed attention, and transformers: https://www.youtube.com/watch?v=yGTUuEx3GkA

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact