> Trying to conflate the issue by suggesting that "we don't know how humans reason, so how do we know AI bots don't reason" is, frankly, absurd. We can easily demonstrate that they are inconsistent and have no concept of what they are writing responses about, as shown above.
The point of comparing it to human cognition is because this reveals that we simply cannot make categorical statements based on how we believe we reason. At our current level of knowledge about the brain and consciousness, it is still a possibility that we are a bunch of neural networks that decode language and, in doing so, produce justification for our actions which, in some contexts, can lead to what you would describe as logical or reasonable output. Sometimes this output is incorrect, and we are definitely not internally consistent. In particular, some of us are very often both incorrect and inconsistent. I doubt you would call a human with an IQ of <60 as incapable of reasoning, for example, and yet I feel such a person would have similar difficulties with most of the tests described in the paper.
So, in short, I would reverse the question here: if your only claim is that AIs don't reason like us, this is a very weak argument in favor of the claim that they are incapable of reasoning.
The point of comparing it to human cognition is because this reveals that we simply cannot make categorical statements based on how we believe we reason. At our current level of knowledge about the brain and consciousness, it is still a possibility that we are a bunch of neural networks that decode language and, in doing so, produce justification for our actions which, in some contexts, can lead to what you would describe as logical or reasonable output. Sometimes this output is incorrect, and we are definitely not internally consistent. In particular, some of us are very often both incorrect and inconsistent. I doubt you would call a human with an IQ of <60 as incapable of reasoning, for example, and yet I feel such a person would have similar difficulties with most of the tests described in the paper.
So, in short, I would reverse the question here: if your only claim is that AIs don't reason like us, this is a very weak argument in favor of the claim that they are incapable of reasoning.