Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Traveling words: a geometric interpretation of transformers (arxiv.org)
80 points by d4rkp4ttern on Oct 5, 2023 | hide | past | favorite | 4 comments


I like this definition. Simple, clean, effective.

Maybe the start of a new field tackling 'travelling wordsman' (smith?) problems.


To anyone knowledgeable: where does geometric deep learning fit in here? Is this paper just another geometric view, or is it an attempt at a formalization for transformer mathematics? I don't see prominent GDL authors in this paper's references (Bronstein, Cohen, Bruna, Veličković, ...).


I think it could be applicable to developing a novel geometric theory if the dynamics of the word trajectories were better understood.


Geometric learning is more about group- and invariant theory. This paper only goes as far as linear algebra, as far as I see.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: