I think in your notation it should have been: y=Wx_0 y=W(x)x_0 | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		oneearedrabbit on May 17, 2023 \| parent \| context \| favorite \| on: Ask HN: Can someone ELI5 transformers and the “Att... I think in your notation it should have been: y=Wx_0 y=W(x)x_0

nil-sec on May 17, 2023 [–]

I guess I was more thinking about self attention, so yes. The more general case is covered by your notation!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact