Aside from the using the word "concept" instead of "language" I don't see how this is different than an LLM. It's still doing next token prediction. This is like in D&D where you have two swords with wildly different flavor text but ultimately they both do 1d6+1 damage.
What am I missing -- aside from the marketing? Is there something architecturally different or what? Looks like regular autoregressive sequence transformer to me.
(Guessing here) It does tokenization and prediction for a whole sentence, not fragments of words.
I like this idea because that's how humans think. We mentally formulate a whole sentence, then say it. People who don't do this speak in run-ons and word salad.
I would be interested to know how many people do formulate a whole sentence before saying it. "Think before you speak" as they say. I feel I do not have the cognitive window or processing speed to do this; instead, I formulate a concept of how I would like to respond abstractly, and then think of and say phrases of several words one at a time until the sentence ends itself. The latter process leans heavily on some kind of next word anticipation.
> how many people do formulate a whole sentence before saying it
The process is a formulation of precise ideas (complex at some level and verified to some degree, hopefully), then translated into sentences for output (not necessarily in these two steps, but through iterations).
This project tries to use sentences as formalizations of ideas - an interesting way enabled by availability of tools, allowing good features like transparency.
oh interesting. concepts as tokens. Yeah I'd buy that. They do something similar with transformers in robotics, except they use tokens as actions instead of word chunks. Good eye.
An embedding space engine accepting sentences (SONAR) fit in so that the tokens of this architecture are complex sets of the tokens of past architectures.
What am I missing -- aside from the marketing? Is there something architecturally different or what? Looks like regular autoregressive sequence transformer to me.