The important thing for discussing this is that this kind of reproduction is irrelevant.
Does homeopathy work? Well *I* went to a local hospital and saw people recovering by drinking water and resting.
Well, yes, of course you did. That doesnt count as a reply. It misses the claim.
The claim isnt that there arent an infinite number of Q/A prompts where A is correct. Of course there are, rather trivially just from the nature of a generative model.
The claim is that the reason A is generated from Q is because of the co-occurant frequency of Q,A in a corpus, and that this is not a model of eg., logical inference, causal reasoning, abduction, etc.
It's trivial to show that with a handful of cases of failure. Success is irrelevant, it always is.
These arent claims about the engineering rigour of chatgpt, theyre claims about how it obtains A from Q.
Which, if the GPT had remembered all useful Q,As for all of humanity so far, wouldn't be detectable by prompting. Indeed, even if this is so, the reason we care is when there are novel Qs.
Eg., "what is the local post office's telephone number?" isnt answerable with all of human history up til 1900.
Does homeopathy work? Well *I* went to a local hospital and saw people recovering by drinking water and resting.
Well, yes, of course you did. That doesnt count as a reply. It misses the claim.
The claim isnt that there arent an infinite number of Q/A prompts where A is correct. Of course there are, rather trivially just from the nature of a generative model.
The claim is that the reason A is generated from Q is because of the co-occurant frequency of Q,A in a corpus, and that this is not a model of eg., logical inference, causal reasoning, abduction, etc.
It's trivial to show that with a handful of cases of failure. Success is irrelevant, it always is.
These arent claims about the engineering rigour of chatgpt, theyre claims about how it obtains A from Q.
Which, if the GPT had remembered all useful Q,As for all of humanity so far, wouldn't be detectable by prompting. Indeed, even if this is so, the reason we care is when there are novel Qs.
Eg., "what is the local post office's telephone number?" isnt answerable with all of human history up til 1900.