At the same time I don't think any of this really addresses the root causes of the problems, which are career, university administrative, and field (i.e., recognition and attention) incentives. The whole paradigm we use, and the way we treat scientists and fund research, is broken. It feels a little bit like bailing water out of your boat with teacup on one side while ignoring the gaping hole letting rushing water in on the other side.
The solution is unfortunately for everyone, more transparency in research. When you conduct an experiment, why on earth is the report composed months, sometimes years later, instead of being streamed live and recorded?
Just the same as in business. We need a new more prestigious stock exchange for companies that are willing to open their books up to continual audit.
The government can step in to aid businesses and researchers that act openly, in the form of grants to researchers or tax breaks for members of the more restrictive stock exchange, and shift the cost of policing destructive fraud to rewards for constructive conduct.
Scientists need to care about the truth and most do care. The problem is both that as "publish or perish" becomes more and more literal people feel a strong incentive to cheat and moreover we have increasing cynicism as the system treats a given scientist as disposable.
Constant surveillance wouldn't address that, if anything it would increase the cynicism. Just much, no brilliant scientist is going to work under constant surveillance so you'd drive the most serious thinkers out of any given field.
Working with a labmate is also constant surveillance. I think you're dressing it up to be something far more negative than it actually is.
I think streaming or recording would decrease cynicism. The openness would allow scientists to avoid false idolatry of people who juiced their citations with fraudulent research. And the cost of dealing with the fallout from fraud would be eliminated. I suspect that the fraud itself is a big accelerant of the hypercompetitive publish or perish phenomenon.
Maybe some funding of a transparent program could be allocated toward severance for scientists who honorably choose to "perish". Of course, perishing in this case means still being able to live as a civilian outside of academia. And that person can still speak to those in charge of research projects and benevolently influence the arc of human understanding, which for most good people is enough of an incentive to continue living and contributing.
Folks who wouldn't want to be transparent in their methodology could still do science the old way, they just wouldn't have access to the extra funding.
I just find it odd that people think folks who allegedly value observation enough to make it their full time job would be against observing themselves.
Today I "conducted an experiment". Which means that I went to a grocery store, did laundry, and took a walk in a park. Between all that, I checked once in a while that things were proceeding as expected. The outcome of that experiment is probably never going to be reported, except maybe if something unexpected happens.
Real science is full of messy exploratory work. It's sometimes done remotely, and often in short stretches regardless of what time and day it is. And you usually don't know in advance if it's going to be relevant.
At some point, you may start believing that you have discovered something other people may care about. Then you start thinking about the normative experiments that are supposed to demonstrate the result and perhaps even validate it. But at that point, most of the work is often already done. The discovery has already been made (or at least you believe so) and you are just trying to replicate it. Maybe you could record those experiments if you are really into it. But the recordings would likely be meaningless without the actual written report.
"Checked that things were proceeding as expected" meaning that there is some process going on that you could be recording?
I don't think it's fair to paint recordings of exploratory work as meaningless. You obviously have some interest in it or else you wouldn't be doing it. If you don't share that experience, wouldn't someone else spend time trying the same thing?
In my case, it was just code running on a single cloud instance. For someone a bit upstream from me, it could have been a commercially made device doing its stuff, with the details being trade secrets of its manufacturer.
Computers have made science boring to watch. Many things are now automated and run 24/7. Even if you happen to be on site, there may not be much to see.
In many situations, automation also makes doing things on your own easier than trying to determine if someone has already done similar experiments. If you are doing exploratory work, there is often no established terminology for the topic. Even if a report of prior experiments exists, it can be difficult to determine if it's discussing the same thing. (AI could plausibly help here.)
No the problem is that there's no incentive to do your job between the PhD and permanent position phase. There's no reward for doing anything except publishing an amazing, mind shattering paper that will get a billion citations yesterday. Because unless you do that, everything else seems to be distributed randomly. So people cheat because the system is inherently unfair.
That's why the incentives exist. Folks who can stomach it get tax breaks or bigger grants. Evidently, we're not able to stop fraud, so it makes sense to analyze how much we're spending fighting fraud, and spend that paying people to be honest.
But how do we “solve” those incentives? There will always be winners and losers, and since successful fraud is by definition indistinguishable from genuine success, there will always be people trying to game the system.
The only effective solution I see is making fraud harder to hide. (There’s also making the penalty for fraud higher and trying to instill integrity, but the problem is some people will still take the risk.)
The major issue with research is how only successful results are viewed as worth while, and study results without a "positive" out come is viewed as a failure. So this means researchers are incentized to p-hack or falsifying results since they need more funding.
The reality is these negative results are just as useful and can either help other researchers avoid dead ends or they might have a not great result but with some refinement to methodology might be worthwhile for further study.
Researchers should be judged on the quality of the research they perform, not their outcomes. A well crafted study that shows that further research shouldn't be done should be just a well received as a well crafted study showing more research should be done. The focus should be on the design of the study first, since no matter what the results it adds to the corpus of knowledge in a meaningful way
You are 100% correct. The problem is, even without the incentive of money and employment there's still the incentive of fame, and negative results simply won't get the same attention and impact as positive results. We can give people with negative results PhDs and publications, we can continue funding them. But at the end of the day, people remember and researchers cite the team who invented the LLM, not the many teams who tried and invented the less-successful models.
Four words: Academic fraud is fraud. If you accept an NIH grant to conduct a rigorous study and falsify your data, then you have defrauded the federal government. If you are a supervisor, a department head or an administrator and you turn a blind eye to implausible results and indefensible practices, then you are complicit in fraud. If your journal claims to be rigorously peer-reviewed but your reviewers are rubber-stamping papers that they haven't read, then the institutions that buy very expensive subscriptions are being defrauded.
Only in academia is fraud treated as a faux-pas rather than a crime. We have accepted a status quo in which academics and academic institutions play by a different set of rules to the rest of society. As a result, hundreds of millions of dollars are being wasted and patients are suffering real harm. Academic independence is valuable and should be protected, but independence does not mean impunity.
One idea is to put more pressure on writers of letters of recommendation. There is obviously already some rather weak "reputational incentive" to not give letters of recommendation to fraudsters. But it could be made a more formal policy. Letters could be required to include a statement about the candidate adhering to various formalized "good behavior" policies. Then when fraud happens there could be actual consequences for the letter writer. This would be like the culture in software development where you don't blame people for writing bad code, you blame reviewers for letting in bad code. (Idea comes from Angela Collier, in relation to other forms of antisocial behavior).
1. There is already a lot of administrative bloat in the academia, and ideas like this are responsible for most of it. Every time you replace an informal process with a formal one, you add to the bloat. And the formal process rarely works as intended.
2. Letter-writers have no control over the work of the person they recommend. They cannot prevent the submission of a manuscript or a grant. They would be taking a lot of risk for no gain.
3. Letters of recommendation are a form of corruption and nepotism. If you would have a conflict of interest for reviewing a manuscript or a grant, you should not be allowed to write a letter of recommendation either.
If what you propose would be implemented, why would anyone ever write a letter of recommendation? I'm currently doing an unpaid favor to the institution which asks for letters of recommendation, as I spend my time so they get my opinion about the candidate. If instead of thanking me for it they would want to put more pressure on me and assign actual consequences, well, that's insulting and I'd pass, I owe them literally nothing.
> since successful fraud is by definition indistinguishable from genuine success
This is absolutely false in real science and that you say it seriously is proof of just how far gone the academy is. Needless to say no matter how much you fake your data or results, nature is not so easily fooled.
We've had these incentives since we started science. Remember Albert Einstein wasn't able to get a teaching position until after he was winning the Nobel. It turns out the people that really love science will do it even if it doesn't pay anything and brings them nothing.
We just need to make sure the salesman scientists and parasitic beuracrats don't squash the efforts of those doing real work.
The root cause it too many people for the funding available. If a single scientist can spend $250k per year it's obvious that even rich economies can only support so many of them.
Whenever you have this situation you inevitably lead to selection criteria which can, and will, be gamed.
It's not just the funding but the jobs themselves. Universities and even the funding bodies want to show shiny easy-to-digest numbers to their stakeholders to justify tuition and government funds. Which means these institutions in turn expect those numbers from their researchers, who then are happy to comply and build their paper mills and citation cartels, solving no real problems at best and producing fake data at worst.
One positive move would be for more institutions to publicly endorse the recommendations of the Declaration on Research Assessment. It won’t solve all of the incentive problems, but it should be a step in the right direction. I wrote about it briefly here a couple of weeks ago [1].
Even if you resolve fraud, you still fail to resolve the driver of the fraud, which is money.
There are basically 3 sources of funding for universities - government, military, corporations. These 3 have aligned interests. Where their interests align (or are not in conflict) you will have funding of one side only. If you fund one side only, you will generate lots of information to support whatever-it-is, but this will fail to be a fair representation of the issue.
I'll just run through a simple, silly example. If I wanted to prove cats cause stormy weather, I could fund say, 10 studies. Perhaps 2 are supportive (cats and storms correlate in some way). I can then fund even more studies in that direction. I would soon have a huge body of supportive data .. but I haven't said anything meaningful, I haven't got to the truth of the matter. I just have a huge body of evidence.
To me, this is chicken feed. It's not that the information is a lie necessarily, it's just that it's isn't important. Nevertheless, the body of data can be referred to, in order to help steer public opinion. Everyone is getting paid, the thing is true .. what's to be upset about?
Anyway, whoever has the money determines the outcome. Truth is a secondary consideration.
One of the many troubling things about recently exposed patterns of fraudulent science: every dishonest scientist or professor is taking up a position and funding that could've instead gone to any of surely numerous people with much more integrity.
Permit all the cronyism and gaming around university hiring and admissions, and don't hold universities responsible for unethical behavior of their faculty and graduates, then maybe this is what you get.
And since this is HN casting stones in glass houses: Permit all tech industry fratbro conventions around hiring (e.g., some that originated as naive and classist "did you take the same Stanford classes that I did", then were cargo-culted), and don't hold the companies responsible for the growth pump&exit investment scams, nor the criminal incompetence around infosec and correctness, then maybe that's what you get there.
I'd go so far as to say that the mere existence of an "exit plan" is an indicator that the purported product may be flawed (the business may not be flawed, from an investor's PoV, but to them, the "product" is actually the business, itself, as opposed to what the business says is their product).
IMO: Real businesses are meant to be ongoing enterprises. Being sold to others, is supposed to be an unplanned (possibly fortunate and hoped-for) event. It shouldn't be an inevitable business plan milestone.
Agreed. The idea that financial engineering is somehow necessary to "promote liquidity" in facilitating the changing of ownership of a business is completely arbitrary, and not inherently good as far as I'm concerned.
Sure, rules that make it harder to buy companies might slow down the volume of businesses being created due to less investor interest, but I'm not sure that decline would surpass the slowdown in fradulent exit schemes.
> every dishonest scientist or professor is taking up a position and funding that could've instead gone to any of surely numerous people with much more integrity.
Maybe, maybe not. It feels possible at this point that the entire top of the competitive curve is actively juicing. Ie, with such fierce competition, there could be a vanishingly small cohort of researchers who both operate with integrity and whose CVs are competitive for faculty positions.
At best, we might expect a removed fraud to be replaced with someone marginally more honest.
To get more good research done, there are at least two different fronts we could fight on. The straightforward one is to reduce incentive to fake at the "top of the competitive curve", somehow. But the total population of researchers is still so small, it might not be the most important front.
The other front is to let the fakers keep faking, but change society so that everyone who wants to research out of personal interest has the free time to do so. They'll have less incentive to fake.
Just as everyone in the Tour de France was doping during the Lance Armstrong era, the concentration of cheaters increases the higher up the "success" metrics you go.
Because cheating without getting caught will always be the "best" strategy within the confines of the game alone.
However, that's likely the worst strategy for society as a whole.
> there could be a vanishingly small cohort of researchers who both operate with integrity and whose CVs are competitive for faculty positions.
I think it's fallacious to compare the CV "competitiveness" of a falsifying 'scientist' with a legitimate scientist.
Not that that's what you were suggesting, but maybe we haven't fully accepted what it means to combine the typical daisy-chained/chain-reaction growth of CV opportunities (one prestigious-looking line item opens a door to one or more others), with cheating offering additional boosts as needed at each step.
When an athlete "juices", they can perform with greater strength or endurance.
But when a scientist falsifies research, they aren't performing greater science -- they're performing greater CV (and terrible science).
Competition for faculty positions shouldn't be CV, but the potential for great science.
If an institution wants to use CV as the proxy for science potential, then they have to address that the CV metric is being gamed like crazy, including by falsifying (i.e., terrible) scientists.
Except that HN does not subsist off of non-volition* fnding towards subject matters which don’t benefit human life. And the ones that purportedly do are so off, we have the evidence of the genetic regression to the mean in the randomly sampled scientists heredity vis-a-vis independence in free will power, I.e. they come from peasants and slaves and lack original insight such as perceiving the world entirely electromagnetically.
The lack of training social scientists in cell life for a bachelors, or the biological sciences in electrical signaling, or the physical scientists in their necessary inferior perspective of natural changes because of measurement imperfections by their instruments and their body’s senses, is all telling for a community which has been avoiding the successes of electrical engineering for over a century, likely because they don’t want to do that math ;)
I really hope these people gain wider support. They seem so anti-establishment, which implies that the establishment is pro fraud. And that sounds about right to me.
This is a huge problem. When fraud is rewarded over legitimate research we lose so many times. Once when we reward the fraud, again when the fraudulent research is propagated into the corpus, again when we recognize that this has squelched legitimate scientific research in exchange for resources being sent to the fraud. And finally when trust in science falters.
There's every chance that the next great scientist is flipping burgers at McDonald's.
The problem is much bigger than anyone wants to admit. In some domains a coin flip is just as good as relying on peer reviewed research.
>This is a huge problem. When fraud is rewarded over legitimate research we lose so many times.
I’d generalize this further and say we have an underlying economy and society that rewards fraud over what should be happening widespread. It’s happening everywhere and lying seems to be one of the best strategies to success. With competitive pressures increasing, it’s often even worth the risk. This may have always been the case but the population at large seems to be more cognizant of it and taking action on it. Politics, business, everywhere deceit is rampant. It may in some cases be “allowed” forms of lying or skirting around the truth but underlying behavior is the same here: avoiding the truth for one’s own self gain. There’s few if any establishments I trust anymore. I just assume everyone everywhere is lying for their own goals.
>> There’s few if any establishments I trust anymore.
Sorry, but that's tin foil hat territory. Everyday I trust my bank to handle my money fairly, the gas station to give the right amount of gas, the labels on food and medicine to be accurate, the weather report, etc. etc. I am sure there are problems with virtually every institution that need to be addressed. But I am surrounded by reliable information and trusted actors.
Society works on trust. That said, my bank/credit card company would not work with us when we were scammed with
$10k of obviously fraudulent charges. Our next recourse will have to be legal action. There are reports of gas stations using more ethanol than labeled. I saw a tiktok last week on someone who brought in their own scale to walmart and the packaged meat was labeled at like 6 lbs but was actually under 5. My tax guy has fucked up 3yrs in a row so now I get to become a tax pro. The US govt owes me $40k and is dragging their feet sending me my money. The guy who repaired some external trim only used finish nails so it ripped out in the last wind storm. The company that installed my windows just ignored code and didn't install tempered glass where required so I had to fight them on that when someone pointed it out to me. I could go on and on.
Yeah we need to inspect things carefully because we are getting scammed and lied to all the time. Olive oil being cut, fish being sold as the wrong species, supplements containing random other substances, there are a million examples in products in particular.
I'm no expert, but I do. There is a lot of rotten olive oil sold in the US because we don't seem to care so much. Smell and taste is the best indicator. I am really irritated that my Italian dressing only contains only rapeseed oil and not olive oil so it's something I've been looking into lately. Olive oil is apparently very volatile and rather complex.
You can totally inspect your olive oil yourself. First a primer:
Extra virgin olive oil is the highest grade. This is unregulated and mostly nonexistent in most US grocery shelves.
Here's the criteria I recommend:
- Source. Most Italian olive oil is repackaged Tunisian, Turkish, Spanish olive oil. Which isn't a bad thing but consider why they're not consuming it themselves (Italy and Greece do not produce enough to meet domestic demand). You'll see this by abbrvs like Source: "TN SP TK" (Tunisia, Spain, Turkey).
Look for single source oils. Consider Californian oil (if in the US).
A country does not produce better olive oil than another, it's all up to individual producers. Some of the best oils I've had are from Australia, California, Chile
- Smell! Does it smell like wax/waxy crayons/Crayola? It's probably a bit rancid/old. Not necessarily bad for you but lacks the health benefits (it's not extra virgin at this point if it ever was). It should smell fruity/grassy/green.
- Taste: It should be bitter, fresh, and tickle your throat (that's the antioxidants!). It shouldn't leave your mouth feeling greasy.
- Harvest date: Oil degrades quickly. If there isn't a harvest date, it's probably not great. Should be harvested within the last 2 years, ideally within 1 year.
Extra virgin has the maximum taste and health benefits. Most grocery store oils are simply olive oil - not the highest grade but still significantly better than seed/nut oils.
Wow! Thanks for all the info. I’m getting the EVO brand California Olive ranch right now. Is this any good? I’m interested in making my own Italian dressing. Any pointers?
California Olive Ranch oils are the only ones I've seen that are consistently decent across the US. I do occasionally see bottles from 2021, so be wary. 2022 is the latest year you should buy, most coastal cities/national Whole Food chains should be starting to stock 2023 now. This is the brand I direct my mother to buy so I have skin in the game :)
BUT BUT BUT make sure it's NOT the "global blend" - they got in a lot of trouble in California for trying to pass it as good oil. It's trash. Blends are usually made to maximize profit which minimizes the flavors and benefits of good oil. Look for their oil from California in the glass bottle.
If you're in California, the Bariani brand is affordable and awesome.
See comment above. Your local olive oil store (if there is one) should have great options. If you're on the east coast, one of those local high-end "Italian" stores may have a knowledgable person who stocks good oils.
Yes and no. E.g., Banking. Yes, trust them for transactions. It's their bread & butter. But if 2007 / 2008 taught us anything it's that the financial sector can't be trusted beyond simple transactions.
Labels on food? For ingredients? Again, sure. But where is the label for "the taste of this product has been engineered to increase your consumption of it. Your health? Ha. Forget about it."
And so on.
You're trusting the sheeps' clothing and are all but ignoring the wolf that hides beneath it.
i actually work at a financial institution and you should not trust your bank, you should review your statements and complain if you find errors. which you will.
> They seem so anti-establishment, which implies that the establishment is pro fraud.
What I'm getting from the article so far is that "the establishment" is actually a bunch of establishments (plural). And thus there are many systems to game, and many incentives to game them.
I agree. Saying that "the establishment is pro fraud" is not helping at all, because conforming with such a simple label will make us blind to the actual reasons why fraud occurs. It is not that all fraudulent scientists are part of some cult or get taught to be fraudulent. The fraud rather results out of a long series of seemingly small decisions that stack up to the point where they hire paper mills, all the while not performing enough self-reflection. If you want to combat this, you will not be successfull labeling the people doing this as fraudulent. Some may take offense to this and not listen to you. Rather, apart from systemic change also more reflection and ethical awareness should be promoted (Please stop thinking and teaching that science has nothing to do with ethics - that just allows scientists to be blind to their own faults).
They seem anti-establishment, because you need an anti-establishment attitude to do investigative work like that. If you are the average conformist, you are happy to work within the system, focusing on your personal goals.
There are two things that you should put into perspective when seeing these numbers.
1- The article give pure numbers and does not report ratios. So when they report
> Last year, more than 10,000 research papers were retracted, marking an all-time high
The source is nature publication and while it gives rate of retraction per year it doesn't show the ratios. So if this is 10,000 out of 5M or 10,000 out of 10M is huge different. Yes in both cases it will be a problem but also there are no information about re-traction cause. There is no statistics on cause of re-traction. Not every retraction is necessarily because of fraud.
Actually in the nature paper they reported the retraction rate (without any other information) to be 0.2% in 2022 but again without more information.
2- calling it epidemic is clickbait title. Depending on retraction as a measurement of fraud problem in science is subjective. There are problems with citations, paper mills and many other things. But you can't put all of these things into one basket. Retraction is a standard process that even legitimate researchers are using sometimes to fix problems with the paper ranging from discovering malfunction device after publication to method not working. Associating retraction with fraudulent activity only would mean that a researcher wouldn't use it if the perception that if you do it yourself then you are trying to cover something or you have ethical problems.
This shouldn't be treated as a mathematical problem. The consequence of just one of this 10.000 articles can lead to thousands being killed or suffering traumatic useless treatments that seem designed by sadists. A single article and people still are told decades later that vaccines cause autism. Other, and parents will start believing that bleach enemas will cure their autistic children. Other and millions of dollars will be lost. Spreads like fire and is simply evil.
And when you'll eventually convince everybody that those ideas are nonsense you'll have still 9997 fake articles more to fix.
Having recently listened to the freakonomics episode on the same thing I'm mystified by something that podcast and this article do. They state that they're is an estimated 2% of bunk papers or there but then also that they are "becoming overwhelmed by the amount of bogus research" or that they "would estimate that there are far more paper mill papers than general papers" when it's like "swimming in garbage".
2% doesn't seem like it should elicit those sentiments.
I feel the article needs to put forward more evidence that it is an epidemic. It looks just as likely that science has had systematic problems for a long time and abuses have always been rife.
If you’re not in it you don’t see it but after seeing sufficiently faked fluorescence studies I believe nothing. On the Internet some expert HN user will tell you how “it’s worth noting that there is some evidence” and it’s probably some faked science on photoshopped fluorescence study. Useless crap in a worm. And it’s not even real.
But you can’t tell anyone this because they’ll circle the wagons. “Oh yeah, you think science is blah blah blah”.
That’s so fundamentally core to the idea of authoring a paper that it would be fundamentally useless. Anyone knowingly publishing fraudulent data will not be deterred by a signature. It’s already career ending if discovered.
I had read GP's link prior to this. It's about a researcher claiming that signing a truthfulness statement beforehand reduced lying, but that very research was tampered with.
It was assumed by default that scientists don't lie (and researchers are signing their articles since thousands of years). Innocent until proven guilty is a better system, but... lets add even more bureaucracy to publish, is not insulting enough yet.
If there are no criminal penalties for the vast majority of academic fraud, it will continue unabated. Similarly, financial fraud is not prosecuted in almost all cases in the USA. And those who do get prosecuted often pay nominal fines if convicted, while they were raking in 8+ figures in personal profits.
Scientific fraud isn't going anywhere, and neither is academic fraud. In fact, it will only continue to worsen as most of the Global North is in a post-Truth, post-meritocratic society. Heck, tenured professors and administrators often will not even be censured except in the most extreme cases, let alone lose their jobs, let alone ever see the inside of a prison cell.
They need to start putting in grant contracta that if the researxh is found to be fraudulent the ibstitution is resppnsible for oaying back the grant money. This would also helpbsolve the endowment problem
Scientific fraud is a culture problem. We need a hard reset on universities - remove grants and FAFSA support for uni's for a few years. Those that survive prove they are tied to industry and aren't just coasting on welfare.
I think the discussions around this issue (in this thread and elsewhere) are missing a key point: The vast majority of academic research is conducted by graduate students working toward their PhD. For instance, the word "student" does appear in this page even once. I am not condoning fraud, but put yourself in the students' shoes, and ask which is more likely:
"Hmmm, these experiments that I have spent several years working on, and are the basis of my entire thesis, didn't work out. If I report the correct results, my entire project will be worthless and I will not earn my PhD. Too bad I wasted 5 years of the prime of my life pursuing this hypothesis which my supervisor insisted I investigate. I hear Costco has good benefits, I wonder if they're hiring?"
or
"Hmmm, if I just delete some outliers, cherry pick some experiments, ignore others, nudge a few of these data point in the right direction, and choose this specific statistical analysis, I can show a positive result, which means I can graduate, start my life and family (finally), get a job (maybe in academia, maybe not), and fulfill my dreams."
As I said, I am not condoning fraud, but the discussion of solutions need to focus on the right problem. Most discussions seem to imagine a professionally established scientist whose prestige and job promotion depend on good results. In reality the stakes are a lot higher if you realize that these results of literally life-changing for the graduate students doing the work. I guess I'm saying there is a human side to this issue beyond greed and prestige seeking (although there is plenty of that too).
The rewards of fraud in this context are orders of magnitude greater than its costs. The article is not wrong when it blames “bad incentives” and “the rampant culture of publish and perish”. The root cause, however, is the steady cultural shift away from honesty, integrity, and other (sine qua non?) values. As a society, we’re way too tolerant of lies and deceit IMHO.
In my opinion, the solution to all of this is to get rid of the whitepaper system all together.
From now on, all scientists who want to publish science, they publish their work in the form of a video. They film themselves doing the science and then put that video on YouTube, or maybe some science specific version of youtube. Other people can then watch the video and determine for themselves if the science is valid or not.
Its much harder to fake science if there is a video involved. Its too easy to make fake science if all you have to do you "prove" it is write some words.
It takes of order 6 months to do the work described in a single paper. Even if we imagine someone watching the videos at double speed, it would take 3 months per paper. And that's for an expert who can understand the material.
Folks don't have the time to read the literature thoroughly as it is. That's one of the reasons the junk gets published. So this is not practical.
Plus, imagine reporting to your funding agency, Dean, or boss, that all you accomplished this year was to watch a few videos. No new work of your own. I guess you would make a video of you watching a video. And then someone would watch a video of you watching a video ...
My guess is that more than half of the world's scientific resources are spent on projects in bad faith, so I'm actually ok with legitimate scientists spending 100% more to justify their papers, if the extra work can be covered by the surplus resources saved by eliminating frauds. However, there are many reasons the proposed method is impractical.
In most theoretical/abstract pursuits, from math to comparative literature, "doing the science" means "writing the text of the paper" - and nothing else; shall I film myself doing that?
On second thought, maybe I could make a video of how I manage all of those dirty LaTeX tricks to squeeze my paper into less pages, at least there's some tangible payoff there...
> "doing the science" means "writing the text of the paper"
The point is to show your work. You film yourself setting up the experiment to prove that you actually did the experiment. You also film yourself collecting the results, proving that you actually got the results you claim.
The only kind of science that consists of "just writing the paper" is math. Math will be exempt from this requirement. Fake math is not an epidemic
At the same time I don't think any of this really addresses the root causes of the problems, which are career, university administrative, and field (i.e., recognition and attention) incentives. The whole paradigm we use, and the way we treat scientists and fund research, is broken. It feels a little bit like bailing water out of your boat with teacup on one side while ignoring the gaping hole letting rushing water in on the other side.