Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I was also taken by the capabilities of chatGPT but with time and with people reporting how it sometimes fails with simple tasks (even v4) I'm beginning to ascribe its superpowers a bit more to the super impressive magnitude of its training data. That isn't to say it can only somehow reflect the data, but even Logan Kilpatrick who is an OpenAI employee said in a youtube video somewhere that chatGPT absolutely can't reason (though there wasn't much of a followup as I recall). I think the problem is precisely as you state - there isn't a reliable way to reproduce the findings because the findings were produced with a bit of a dice roll, to put it simply. This also happened which Chomsky when he weighed in on the subject and some people pointed out his findings don't match theirs.


The important thing for discussing this is that this kind of reproduction is irrelevant.

Does homeopathy work? Well *I* went to a local hospital and saw people recovering by drinking water and resting.

Well, yes, of course you did. That doesnt count as a reply. It misses the claim.

The claim isnt that there arent an infinite number of Q/A prompts where A is correct. Of course there are, rather trivially just from the nature of a generative model.

The claim is that the reason A is generated from Q is because of the co-occurant frequency of Q,A in a corpus, and that this is not a model of eg., logical inference, causal reasoning, abduction, etc.

It's trivial to show that with a handful of cases of failure. Success is irrelevant, it always is.

These arent claims about the engineering rigour of chatgpt, theyre claims about how it obtains A from Q.

Which, if the GPT had remembered all useful Q,As for all of humanity so far, wouldn't be detectable by prompting. Indeed, even if this is so, the reason we care is when there are novel Qs.

Eg., "what is the local post office's telephone number?" isnt answerable with all of human history up til 1900.


I think GPT4, and similar LLM models are the future of search engines, but are not intelligences in a more sci-fi sense, and we are unknowingly far from that.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: