> Do we have a retired BBC reporter that was based in an East European country during the Cold War?
> (...) None of the above challenging semantic understanding functions can be ‘approximately’ or ‘probably’ correct – but absolutely correct.
There already exists systems that do text to sql translation. The question author uses as an example is actually quite easy, and not where modern text to sql systems tend to fail. The "absolutely correct" part is clearly false. Both humans and ML systems will make mistakes and have to deal with ambiguity. As a software developer working with nontechnical stakeholders, I'm 100% convinced that people routinely compose sentences they do not understand in any absolute sense.
2.
Good way to check if your proof is correct, is to check if it leads to absurd conclusions. His proof related to learnability (ML) and compressibility (COMP) would seem to indicate that either people can not learn or they do not understand language. Absurd.
3.
> In ML/Data-driven approaches there is no type hierarchy where we can make generalized statements about a ‘bag’, a ‘suitcase’, a ‘briefcase’ etc. where all are considered subtypes of the general type ‘container’. Thus, each one of the above, in a purely data-driven paradigm, are different and must be ‘seen’ separately in the data.
This is objectively false. Even simple context free embeddings like Word2Vec will capture some relationships between a ‘container’, a ‘bag’, a ‘suitcase’, a ‘briefcase’. Based on that alone the rest of that argument falls apart. But large language models go beyond context free embeddings.
4.
The last argument is relatively good one. It's hard to extract all attributes of human concepts from text alone. This is where large language models fail miserably. We need to provide models access to different modalities, but not only. The models can not be embedded in a static world of recordings in order to create rich understanding of things like agent or agency etc.
This is indeed missing, but at the same time this is being actively worked on.
So, that's it. Author makes 3 either false or absurd arguments. And one that is good, but wholly unconvincing as to impossibility of ML solving natural language understanding.
1. You are conflating the sense in which humans may arrive at a mistaken propositional model based on mistaking context; with one where the machine lacks any sense of contextual relevance to arrive at any specific propositional model.
This tactic is taken in these "replies" often: humans fail for semantic reason A; machines fail for non-semantic reason B; isnt A just B? No.
2. Or you've misunderstood how humans learn.
Though on the face of it the sketch of the proof its correct: there are an infinite number of target models (T) which compress to representation R. Eg., an infinite number of 3D geometries which can produce a given 2D photograph.
Compression (ie., "low-rank" interpolation through data) yields a function from R-space datasets (eg., 2D photos) to a model R which "covers" that space.
It does not yield a function from R->T, which doesn't exist as a formal matter. You, at least, need to add information. This is what many misunderstand: light itself is ambigious and does not "contain" sufficient information. We resolve light into 3D models by a "best guess" based on prior information.
So we require, at least (R, C) -> T where 'C' is some sort of contextual model which bridges the infinities between R and T.
Since ML takes Samples(T -> R) -> R, and not (R,C)->T, it doesnt produce what is required.
QED.
3. Word2Vec does not capture hierarchical relationships. He chose hierachical specifically because it is a discrete constraint and ML is a continuous interpolation technique that cannot arrive at discrete constraints.
4. "Actively worked on" means building AGI. Participating in the world with people is how animals acquire relevance, context, etc.
1.
> Do we have a retired BBC reporter that was based in an East European country during the Cold War? > (...) None of the above challenging semantic understanding functions can be ‘approximately’ or ‘probably’ correct – but absolutely correct.
There already exists systems that do text to sql translation. The question author uses as an example is actually quite easy, and not where modern text to sql systems tend to fail. The "absolutely correct" part is clearly false. Both humans and ML systems will make mistakes and have to deal with ambiguity. As a software developer working with nontechnical stakeholders, I'm 100% convinced that people routinely compose sentences they do not understand in any absolute sense.
2.
Good way to check if your proof is correct, is to check if it leads to absurd conclusions. His proof related to learnability (ML) and compressibility (COMP) would seem to indicate that either people can not learn or they do not understand language. Absurd.
3.
> In ML/Data-driven approaches there is no type hierarchy where we can make generalized statements about a ‘bag’, a ‘suitcase’, a ‘briefcase’ etc. where all are considered subtypes of the general type ‘container’. Thus, each one of the above, in a purely data-driven paradigm, are different and must be ‘seen’ separately in the data.
This is objectively false. Even simple context free embeddings like Word2Vec will capture some relationships between a ‘container’, a ‘bag’, a ‘suitcase’, a ‘briefcase’. Based on that alone the rest of that argument falls apart. But large language models go beyond context free embeddings.
4.
The last argument is relatively good one. It's hard to extract all attributes of human concepts from text alone. This is where large language models fail miserably. We need to provide models access to different modalities, but not only. The models can not be embedded in a static world of recordings in order to create rich understanding of things like agent or agency etc.
This is indeed missing, but at the same time this is being actively worked on.
So, that's it. Author makes 3 either false or absurd arguments. And one that is good, but wholly unconvincing as to impossibility of ML solving natural language understanding.