No Scientific Method Can Generate Integrity

The frequency of articles about misconduct, fraud and reproducibility show that scientists’ integrity cannot be assumed by a “scientific method.”

Scientists are only people. They are not immune to the temptations and failings of others. Peer review and the “scientific method” (if there is such a thing) can guard against some misinformation getting out, but no method is immune from character defects. All conclusions from data must pass through fallible human beings. The following reports show that problems of scientific integrity loom large, despite a method that is widely thought to protect against them.

Another High-Profile Fraud

Vaccine researcher Dong-Pyou Han doesn’t look happy in the photo in a Nature piece about the latest fraud.

Rare is the scientist who goes to prison on research misconduct charges. But on 1 July, Dong-Pyou Han, a former biomedical scientist at Iowa State University in Ames, was sentenced to 57 months for fabricating and falsifying data in HIV vaccine trials. Han has also been fined US$7.2 million and will be subject to three years of supervised release after he leaves prison.

His case had a higher profile than most, attracting interest from a powerful US senator. Han’s harsh sentence raises questions about how alleged research fraud is handled in the United States, from decisions about whether to prosecute to the types of punishments imposed by grant-making agencies.

Is this proof science can keep its own house clean? Not necessarily. He was caught, yes—but only after years of deceit. “Han said that he began the subterfuge to cover up a sample mix-up that he had made years before.” How many others get away with fraud? Harsh as this sentence seems, some feel it isn’t harsh enough. “This seems like a very light penalty for a doctor who purposely tampered with a research trial and directly caused millions of taxpayer dollars to be wasted on fraudulent studies,” Senator Charles Grassley (R, Iowa) complained. He also said, “I worry that other cases may go unnoticed or unaddressed if there isn’t a public outcry.”

The ORI (Office of Research Integrity) appears about as effective at stopping fraud as the U.N. is in stopping terror. They probably wouldn’t have investigated this case at all without Grassley’s insistence, the article says. It ends on a depressing note: “once” when some in the ORI sought to conduct a formal investigation of the impact of penalties for fraud, the Obama Administration shut it down on the grounds that “it cost too much and people were unlikely to respond.”

Reproducibility crisis

“Experiments should be reproducible,” Finagle’s Rules jest. “They should all fail the same way.” The lack of reproducibility in preclinical trials is no laughing matter, three scientists reported last month in PLoS Biology:

Low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development. An analysis of past studies indicates that the cumulative (total) prevalence of irreproducible preclinical research exceeds 50%, resulting in approximately US$28,000,000,000 (US$28B)/year spent on preclinical research that is not reproducible—in the United States alone.

That’s shocking. One might expect a few percent of reports to have problems, but why are over half irreproducible? Is it from carelessness? Is it fraud? This is affecting patients’ lives and hopes. The watchdogs are not sure of all the causes, but “one fact remains clear: the challenge of increasing reproducibility and addressing the costs associated with the lack of reproducibility in life science research is simply too important and costly to ignore.”

The flaw is not limited to the life sciences. Nature wrote about the reproducibility crisis in cosmology. People are getting tired of astronomers “crying wolf” too often about spectacular findings, Jan Conrad writes:

The past few years have seen a slew of announcements of major discoveries in particle astrophysics and cosmology. The list includes faster-than-light neutrinos; dark-matter particles producing γ-rays; X-rays scattering off nuclei underground; and even evidence in the cosmic microwave background for gravitational waves caused by the rapid inflation of the early Universe. Most of these turned out to be false alarms; and in my view, that is the probable fate of the rest….

I also worry that false discoveries are undermining public trust in science. As cosmic phenomena come and go — not to mention endless speculation about hypothetical concepts such as parallel and holographic universes — why should anyone believe that any scientific result will hold?

Conrad does not say that any  “method” can solve this. Instead, “To avoid further weakening of scientific standards and reputations, researchers need to stick to scientific best practice.” How, though, does that advice differ from any other field of human inquiry? “Best practice” is a moral judgment, not a methodological one. Critical examination, alertness to personal biases, rewards and punishments—these best practices are not limited to science. None of them, further, will work without people of integrity.

Maybe this is just limited to a few fields, like cosmology and medicine. Nope, says Stuart Buck, a specialist in research integrity from Texas. He writes in Science Magazine, after pointing out famous cases of research fraud, “Nearly every field is affected, from clinical trials and neuroimaging, to economics and computer science.” Even if he’s right that “Most scientists aspire to greater transparency,” the point is that no “scientific method” protects against immorality. And even for the righteous, “if being transparent taps into scarce grant money and requires extra work, it is unlikely that scientists will be able to live up to their own cherished values.” It’s a familiar human foible in all walks of life, not just science.

Calls for Integrity

This headline from Carnegie Mellon sounds noble: “Top Scientists Call for Improved Incentives to Ensure Research Integrity.” Isn’t science supposed to be a “self-correcting process”? The article calls that a “notion”. It is belied by the problems discussed further down.  It may be comforting to know that the National Academy of Sciences (NAS) is making this a priority, but one might ask, “Who’s watching the watchers?” If the NAS guards lack integrity, it could become a conspiracy or racket. Will a simplistic “Scientific Method” solve this?

“Science is littered with irreproducible results, even from top places, and it’s a widespread problem that looks different in different domains, but there are shared commonalities,” said CMU’s Stephen E. Fienberg (pictured right), the Maurice Faulk University Professor of Statistics and Social Sciences. “As a statistician, I understand how the role of data is critical. But determining how to set a policy to support data access is very complicated — there is not a simple set of rules.”

The authors’ advice is littered with moral words: integrity, responsibility, ethics. “Additionally, universities should insist that their faculty and students are educated in research ethics; that their publications do not feature honorary or ghost authors; that public information officers avoid hype in publicizing findings; and suspect research is promptly and thoroughly investigated.” Can you crank any of that out of a scientific method?

For those who idolize the Scientific Method as the pure, unadulterated path to Truth, study carefully what these articles are saying. Any truth claim must test itself (hear Nancy Pearcey on this). You can’t use the “scientific method” (if there is such a thing) on the scientific method. A method is assumed, not tested. Someone might counter, “Sure you can. You can test all the truth claims that come out of the scientific method against those that are not produced by the scientific method.” But what if the meta-researcher is a fraud? Then you would need another experiment to test the scientific-method-of-the-scientific-method against the scientific method. This creates an infinite regress of watchers watching the watchers watching the watchers. Nothing works without integrity — whether for the individual researcher or for the scientific community.

The scientific method (if there is such a thing) is downstream of character; it is utterly useless without integrity. For integrity, there is no method. There is only obedience to a standard, such as “Thou shalt not bear false witness” and “Thou shalt not covet.” You cannot have science without the Ten Commandments. Science without an eternal guiding principle of ethics degenerates into majority rule, evolving standards, and whatever a given society is willing to tolerate at a given time, or what is imposed on them by people in power (think 1984).

So Who watches the watchers? “I am the Lord thy God; thou shalt have no other gods before Me.” This is the God who makes science possible. We are all accountable to our Maker.  Want to have a robust science producing reliable results? Guide on the words of the Author of the Ten Commandments.

Resource: Listen to Dennis Prager on why the Ninth Commandment is essential for a healthy civilization (including science).