Agree with everything you said, but I want to dig into why I don't find the critique compelling.
In short, I see a need for us all to inherently distrust anything single thing we read -- everything needs to be verified and checked out across multiple (hopefully diverse) sources of information before it can be considered "truth" in any sense. I do develop trust in certain sources over time, but meta-sources like an AI or a search engine don't get this treatment because their scope is too broad. As an example, I tend to trust folks like Fabien Sanglard[0], Eric Goldman[1], and Adam Langley[2].
Example: over on the Morrowind subreddit a few years ago, an artist had been painting landscapes of the game "Morrowind" (oil on canvas, IIRC), and when he went to print them as posters to sell to fans, the site he wanted to print them on warned him that printing his own paintings of vistas from Morrowind was copyright infringement (it's not). He came to the Morrowind subreddit for help because he'd been actively misled.
Sites like plagiarism.org[3] confidently say false things:
> But can words and ideas really be stolen?
> According to U.S. law, the answer is yes. The expression of original ideas is considered intellectual property and is protected by copyright laws, just like original inventions. Almost all forms of expression fall under copyright protection as long as they are recorded in some way (such as a book or a computer file).
This is confidently incorrect in a couple of different ways. It might seem like incompetence (and may be), but it's worth noting that plagiarism.org is actually run by TurnItIn, which profits from people not really understanding how copyright works.
Similar example (I tend to look into copyright, trademark, and patent issues a lot): I got into a discussion with another HNer about IP law, and they cited an article written by an actual lawyer[4] on a site called "The National Law Review" that's simply wrong about IP law to defend their position. It's tragic, because we've built this sort of cargo cult around "citation, please?" but then all-too-often don't keep our critical thinking skills engaged as we evaluate the source.
This is true all over. I've read HN daily for more than 10 years. Not only does every comment need scrutiny (even if stated confidently), but most stories linked on the front page do as well. It's not just blog posts or opinion pieces; some scientific studies that are posted get torn apart in the comments, to enough of a degree that, after cross-checking, I tend to think the study itself is deeply flawed.
And there are numerous stories about how Google's Quick Answers confidently give the wrong answer, as highlighted in 2017 by the WSJ[5].
I'm not saying accuracy doesn't matter, but I feel the "ChatGPT is confidently wrong" needs more context or comparative analysis around it before it becomes a compelling argument, since "confidently wrong" applies to search engines and humans as well. I haven't seen any in-depth studies on this, but would love to.
"Similar example (I tend to look into copyright, trademark, and patent issues a lot): I got into a discussion with another HNer about IP law, and they cited an article written by an actual lawyer[4] on a site called "The National Law Review" that's simply wrong about IP law to defend their position"
The problem with "citation, please" is it only works if the person you are engaging with is acting in good faith.
If I type in "Can intellectual property rights expire if not litigated?" That web site (national law review) doesn't appear in the top 10 results. So I'd question where did they get that link?
Not so long ago I got in a debate with someone here whether muscle exercises can lead to long term increases in testosterone levels.
The person I debated with provided a link to an abstract of a study that didn't look quite right. There was no link to the actual study.
Instead of engaging with them I confirmed that, within a minute or two, I was able to locate a high quality, free to read, medical literature review on the topic.
Maybe my ability to type keywords into a search engine is superior, but instead I came to the conclusion the other poster was half assedly looking for search results to "win the argument" rather than engaging due to intellectual curiosity.
Absolutely agreed. I see so much bullcrap being peddled by people on hackernews and reddit it's absolutely maddening. People hide behind having a source like it's an immovable shield that protects them against having to perform their own critical thinking.
In short, I see a need for us all to inherently distrust anything single thing we read -- everything needs to be verified and checked out across multiple (hopefully diverse) sources of information before it can be considered "truth" in any sense. I do develop trust in certain sources over time, but meta-sources like an AI or a search engine don't get this treatment because their scope is too broad. As an example, I tend to trust folks like Fabien Sanglard[0], Eric Goldman[1], and Adam Langley[2].
Example: over on the Morrowind subreddit a few years ago, an artist had been painting landscapes of the game "Morrowind" (oil on canvas, IIRC), and when he went to print them as posters to sell to fans, the site he wanted to print them on warned him that printing his own paintings of vistas from Morrowind was copyright infringement (it's not). He came to the Morrowind subreddit for help because he'd been actively misled.
Sites like plagiarism.org[3] confidently say false things:
> But can words and ideas really be stolen?
> According to U.S. law, the answer is yes. The expression of original ideas is considered intellectual property and is protected by copyright laws, just like original inventions. Almost all forms of expression fall under copyright protection as long as they are recorded in some way (such as a book or a computer file).
This is confidently incorrect in a couple of different ways. It might seem like incompetence (and may be), but it's worth noting that plagiarism.org is actually run by TurnItIn, which profits from people not really understanding how copyright works.
Similar example (I tend to look into copyright, trademark, and patent issues a lot): I got into a discussion with another HNer about IP law, and they cited an article written by an actual lawyer[4] on a site called "The National Law Review" that's simply wrong about IP law to defend their position. It's tragic, because we've built this sort of cargo cult around "citation, please?" but then all-too-often don't keep our critical thinking skills engaged as we evaluate the source.
This is true all over. I've read HN daily for more than 10 years. Not only does every comment need scrutiny (even if stated confidently), but most stories linked on the front page do as well. It's not just blog posts or opinion pieces; some scientific studies that are posted get torn apart in the comments, to enough of a degree that, after cross-checking, I tend to think the study itself is deeply flawed.
And there are numerous stories about how Google's Quick Answers confidently give the wrong answer, as highlighted in 2017 by the WSJ[5].
I'm not saying accuracy doesn't matter, but I feel the "ChatGPT is confidently wrong" needs more context or comparative analysis around it before it becomes a compelling argument, since "confidently wrong" applies to search engines and humans as well. I haven't seen any in-depth studies on this, but would love to.
[0]: https://fabiensanglard.net/
[1]: https://blog.ericgoldman.org/
[2]: https://www.imperialviolet.org/
[3]: https://www.plagiarism.org/article/what-is-plagiarism
[4]: https://www.natlawreview.com/article/enforce-your-intellectu...
[5]: https://www.wsj.com/articles/googles-featured-answers-aim-to...