> This is kind of akin to that artificially absurd example of "on average, every human on Earth has one testicle".
No. If I claim everyone has 0.5 testicles, that's bad science. If I randomly choose a large enough number of subjects and report that approximately 50% have two testicles and approximately 50% have no testicles, with error bars, p-values, etc., saying that you don't trust this because I am trying to make a career and you prefer to follow your own experience, which after looking between your legs clearly shows that 100% of people have two testicles, would be quite stupid. If I chose only 3 people, or all of them are male or female, or I make any other mistake, point the mistake, but don't attack me personally, and much less science in general.
For example, I followed the link to the study by the Nielsen Norman Group and then the reference to the study of Piepenbrock. They explain well their sampling method, with different groups by age and depending on vision problems. You can clearly see the individual results and the variance, and it is obvious that generalizing to the 100% of population would be wrong, but there are some very clear trends. Calling these researchers "someone trying to p-hack their way to a publishable result that's sensational enough to advance their career" without any proof whatsoever is insulting.
To be clear, I do not intend that anyone changes habits because of these studies. I agree that this is subjective enough to make it a personal decision. But as a scientist trying to make a career, I found the above comment very disrespectful.
I apologise then. I didn't mean to insult scientists. I was more trying to point out the difference between The Scientific Method and Academic Practice.
Especially in the social sciences, there's been a whole discussion recently about how our current system of evaluating and rewarding scientists is not benefitting Science. There's even been high-profile commentators disputing whether the social sciences are actually Science at all.
As you're a scientist I won't bother explaining this to you. I'm sure you're aware of the problems here.
So my point is that for a study like this, there's lots of room for playing statistical games in order to achieve a more "sensational" result that is more publishable and more likely to get cited. We know this happens and we know this is especially rife in this area of study. So I have become much more sceptical of social-science studies showing broad generalised results about a subject applying to the whole human race. Especially if those studies contravene some commonly-held view about the subject. My default position has moved from "well, they know what they're doing so there must be something to it", to "I'm going to assume that they p-hacked their way to a sensational result until I have evidence to prove otherwise".
I might be wrong in taking that stance. I will change it if I have better evidence.
Fair enough, and apologies accepted. I agree that many social studies lack rigor. And, unfortunately, it also happens quite a lot with more technical topics in which it is much easier to be objective.
I get easily triggered when science is presented as a matter of faith, but in fact I totally agree with your skeptical point of view.
No. If I claim everyone has 0.5 testicles, that's bad science. If I randomly choose a large enough number of subjects and report that approximately 50% have two testicles and approximately 50% have no testicles, with error bars, p-values, etc., saying that you don't trust this because I am trying to make a career and you prefer to follow your own experience, which after looking between your legs clearly shows that 100% of people have two testicles, would be quite stupid. If I chose only 3 people, or all of them are male or female, or I make any other mistake, point the mistake, but don't attack me personally, and much less science in general.
For example, I followed the link to the study by the Nielsen Norman Group and then the reference to the study of Piepenbrock. They explain well their sampling method, with different groups by age and depending on vision problems. You can clearly see the individual results and the variance, and it is obvious that generalizing to the 100% of population would be wrong, but there are some very clear trends. Calling these researchers "someone trying to p-hack their way to a publishable result that's sensational enough to advance their career" without any proof whatsoever is insulting.
To be clear, I do not intend that anyone changes habits because of these studies. I agree that this is subjective enough to make it a personal decision. But as a scientist trying to make a career, I found the above comment very disrespectful.