Personally, I'm not bothered very much by LLM confabulation, as long as it's the result of missing context. In most practical tasks, we either give context to the model, or tell it to find it itself using the internet. What I am concerned with is confabulation that contradicts available in-context information, but that doesn't seem to be what is measured here.
Well, the famous Turing test was evidently insufficient. All that happened is that the test is dead and nobody ever mentions it anymore. I'm not sure that any other test would fare any better once solved.
Still, attributing that progress to "years of research at Google" alone is simplifying the facts to the point of being just plain wrong. That kind of research was always very much in the open and cooperative, with deep levels of standing-on-shoulders.
Attention e.g. was developed by Dzmitry Bahdanau et al. (those being Kyunghyun Cho and Yoshua Bengio) in 2014 while interning at the University of Montreal.
The insight of the paper you point to was that with attention you could dispense of the RNN that attention was initially developed to support.
Also nobody in his right mind uses lookup tables where the table value is actually the float approximation of the true f(x) - you choose the support values to minimize an error (e.g. mse) of a dense sampling of your interpolated value over x (or, in the limit, the integral of the chosen error function between the true curve and the interpolation of your supports). If you want to e.g. approximate a convex function using linear interpolation, all the tabulated values f'(x) would be <= the true f(x).
The true value is far more useful in a lot of cases. If you're building a table indexed by the upper mantissa bits of the float, for example, it's difficult to distribute the error properly across all intervals.
I'm sure you could even let every voter verify that their vote has been registered correctly.
Edit: But as a comment somewhere else in the tree noted ,,And if it could tell you that then a third party could force you to reveal that you voted "right" as agreed before.`` - I guess everything's trade-offs.
reply