> Hopefully this can help address the replication crisis[0] in (social) science.
I think it isn't just p-hacking.
I've participated in a bunch of psychology studies (questionaires) for university and I've frequently had situations where my answer to some question didn't fit into the possible answer choices at all. So I'd sometimes just choose whatever seems the least wrong answer out of frustration.
It often felt like the study author's own beliefs and biases strongly influence how studies are designed and that might be the bigger issue. It made me feel pretty disillusioned with that field, I frankly find it weird they call it a science. Although that is of course just based on the few studies I've seen.
> the study author's own beliefs and biases strongly influence how studies are designed
While studies should try to be as "objective" as possible, it isn't clear how this can be avoided. How can the design of a study not depend on the author's beliefs? After all, the study is usually designed to test some hypothesis (that the author has based on their prior knowledge) or measure some effect (that the author thinks exists).
There is a difference between a belief and an idea. I might have an idea about what causes some bug in my code, but it isn't a belief. I'm not trying to defend it, but to research it. Though I have met people who do hold beliefs about why code is broken. They refuse to consider the larger body of evidence and will cherry pick what we know about an incident to back their own view conclusions.
Can we recognize the beliefs we have that bias our work and then take action to eliminate those biases? I think that is possible when we aren't studying humans, but beliefs we have about humans are on a much deeper level and psychology largely doesn't have the rigor to account for them.
Which is a great idea if we ignore all other issues in academia, e.g. pressure to publish etc. Taking such a hard-line stance I fear will just yield much less science being done.
Psychology is IMO in the state alchemy was before chemistry. And there's no guarantee it will evolve beyond that. Not unless we can fully simulate the mind.
I think it isn't just p-hacking.
I've participated in a bunch of psychology studies (questionaires) for university and I've frequently had situations where my answer to some question didn't fit into the possible answer choices at all. So I'd sometimes just choose whatever seems the least wrong answer out of frustration.
It often felt like the study author's own beliefs and biases strongly influence how studies are designed and that might be the bigger issue. It made me feel pretty disillusioned with that field, I frankly find it weird they call it a science. Although that is of course just based on the few studies I've seen.