N=2408 members of the Danish military or mechanical Turk participants, in a paper with one citation but a huge amount of retweets, in the time of the replication crisis. How is this paper credible?
In the replication crisis, psychology and sociology papers replicated about 30-35% of the time IIRC. In your experience, do random internet comments have a higher or lower chance of being true?
As a random internet commenter, I think I face a bit of a paradox when answering this question. Do you, a random internet commenter, think this is a good paper? Should I believe you and the other people here? Should I look at how many upvotes this has on HN and see it as a sign of credibility, when all the upvotes come from random internet people?
I think evaluating credibility by placing a paper in the evidence pyramid is generally a good idea. I don't think the replication crisis is a reason to outright dismiss a paper, but certainly skepticism before replication is always warranted. I think one should also always be skeptical of how universal one's own experience truly is.
I don't know. It features thousands of data points from different groups in two different countries showing the same trend. That's substantially more compelling than anecdotes IMO, even after tempering expectations due to the replication crisis, but to each their own.
The only reason we're seeing it is because a largely untrained audience decided to upvote it, because it matches their anecdotal experience enough that they think it's a good paper.
As another random internet commenter, I anecdotally find that the least tolerant people I know are indeed substantially dumb.
Thus, whatever probability you assign, between this comment and the original equally valid one contradicting it, we can derive no information, and must therefore rely on the study as the tiebreaker, however suspect we find it.
Some of the (outwardly?) most tolerant people I know are quite dumb, and I know many extremely intelligent people capable of tremendous feats of abstract thought who are both very tolerant and very intolerant.
This is an unreplicated social science paper. It’s worth approximately the paper it isn’t printed on.
