That's certainly one way to do it, but where would you place them?
No, but seriously, you could imagine what we witnessed playing out in a high stakes Tom Clancy or Michael Crichton style fable.
The fiery blowhard Pentagon chief, the arrogant know it all tech bro lab head, an alarm being called in from a remote office and surfaced through Amazon.
This seems to be a good middle ground then. It allows for a way to prevent political projects getting grants under the guise of "scientific research", at least when they directly oppose the voters. I don't see any push to defund basic research, and if politicians start doing that there's at least a way for people to voice their disapproval through voting.
Aside from that, so much money was wasted on Alzheimer's research based on fraud.
Instead of programming something yourself now you'll be forced to program through the interface of someone that doesn't speak English putting what you say into an LLM. We've peaked.
> but there’s no denying that the barrier to entry has fallen significantly.
The barrier to entry to make slop is lower, but it's gotten much higher for developing the skill of programming. There was already an issue with a lack of mentorship and path for juniors when agile attempted to turn software engineers into assembly line workers, among other issues with the industry becoming hyper short-term focused.
Now you have educational barriers where students are competing with other students that are cheating with LLMs. There are psychological barriers with learned helplessness. The 100k lines of vibecoded slop produced hits a wall but they've gained no understanding of the code in the process or ability to make changes themselves. At the first job juniors and interns get they're being told not to take the time to learn and understand the problem they're working and instead they need to hit the LLM slot machine or risk getting fired.
> Clearly the theory that LLM's can't "extrapolate" is woefully incomplete at best (and most likely simply incorrect).
What example is there where an LLM has extrapolated? All I've seen is a data set so large and an extra decomposition process making it so interpolation feels like extrapolation if you don't look close enough.
> but a theory of why further advancements can't solve the deficiencies
Newton did it at 23 and there would have been very few people with mathematical training. The LLM would be trained on the entirety of recorded human knowledge and mathematics up to that point, and would get to use a lot more energy so it still has a massive material advantage over young Isaac. Yet I don't believe calculus would magically appear in its response.
A good way to look at it is to compare it to today: LLMs are already trained and are operationalizing a lot more mathematical knowledge than any human, including experts.
Why are they not coming up with paradigm shift in knowledge expression/discovery like humans did back then?
LLMs have been trained on a lot more data than any single human (text wise at least) for years now and these sort of results have only been possible for the latest crop of models in the past few months. Models get better as they get better.
The argument is whether models of today, suitably trained on pre-17th century data (if comparable quantity was available) would be able to "invent" calculus et cetera.
If we believe today's models are sufficiently capable to have been able to do so, why are we not getting these types of results today compared to the entire world knowledge and especially math?
Are research mathematicians simply not prompting LLMs in the right way?
> What is preventing AI from continuing to improve until it is absolutely better than humans at any mental task?
No matter how much compute time it's given to combine training samples with each other and run through a validation engine it will still be missing some chunk of the "long tail". To make progress in the long tail it would need to have understanding, and not just a mimicry of understanding. Unless that happens they will always be dependent on the humans that they are mimicking in order to improve.
> What is the difference between what LLM's do and "true" understanding?
The thing where you can understand the meaning of this sentence without first compiling a statistical representation of a 10 trillion line corpus of training data.
When you think about the word apple and what it signifies, what do you experience? Is there a feeling of "appleness"? Do you think that sense of meaning is equivalent to the numerical weights of an LLM?
When you think about the word apple and what it signifies, what do you experience?
So I have all sorts of associations with "apple" and spent a little time playing with it.
First in a raw physical sense I can imagine an apple in my head, spin it around, imagine its physics with near cylindrical symmetry etc. A red apple is what first pops into my head, although of course I know there are many apple variants and have opinions on their taste etc.
There are many cultural associations I have with apples from Newton to George Washington. The company Apple has its own set of ideas that I interact with when I hear the word.
In other words I can think of various associations I have to the word apple of various strengths. These associations and strengths are functions of my experience encountering the word and actual apples.
Is there a feeling of "appleness"?
I don't really know what this would mean. I would say no, unless it can perhaps be defined what appleness means and feels like. I don't really notice any strong set of emotions or feelings from this thought exercise.
Do you think that sense of meaning is equivalent to the numerical weights of an LLM?
Again I think I would need a definition of "sense of meaning". I don't seem to derive a singular pointlike meaning when contemplating a singular word. I never was contending that human and LLM cognition are exactly equivalent, but I could see these association strengths being represented in LLM weights. I would say then if an LLM has similar association strengths with "apple" then it "understands" apples as well as I do. Of course this is really hard to test, but frontier models could give you all sorts of apple facts and cultural associations and so on. It may slip up and hallucinate, and I'm sure that I also believe at least one false thing about apples.
So what is your brightline between LLM and human understanding in this example? I assume that your line of reasoning would argue that LLMs do not understand apples. Why don't LLMs understand the word "apple?
It sounds like you don't have the subjective experience of meaning that most humans do, so maybe that would explain why you don't think there is anything beyond associations. Maybe this is the core difference that's determining how people see LLMs.
I'm not sure how I would convey what meaning and understanding is to someone if they don't experience them. This is my poor attempt though: There can not just be associations there need to be "things" to associate between. Otherwise you have no ground, it is all map and no territory. Ultimately it would just be meaningless associations between meaningless symbols.
A pair of bolt cutters should do.
reply