Don’t Follow the (Bad) Science

The literature of science is littered with retracted papers,
lies, bias, embarrassments and bad ideas that refuse to die

 

The new editor of PNAS (the Proceedings of the National Academy of Science, one of America’s premiere science journals) found a horror show. when she took on the job. May R. Berenbaum walked into the vast corpus of scientific literature and found it crawling with zombies, cobwebs and aging mutants. And she couldn’t clean it up! On August 10, 2021, she opened the doors for the rest of us to see in an Editorial in PNAS titled, “On zombies, struldbrugs, and other horrors of the scientific literature.”

When I signed on as Editor-in-Chief of PNAS, I had no idea that killing zombies would be part of the job. These zombies aren’t the spirits of Haitian mythology doomed to roam the earth in thrall to sorcerers and shamans or the brain-eating undead of contemporary movie and television fame. Rather, in scientific publishing, the phrase “zombie literature” refers to papers, deemed invalid for any number of scientific reasons, that are retracted by the journals that published them yet continue to be cited without any apparent acknowledgment of their lack of validity.

A creepy sight indeed. Only insiders and subscribers to the journal had access to this confessional, until we decided to share this.

Zombie literature is only one category of difficulties of her new job. Berenbaum lists other examples of bad science infecting scientific literature all the way back to the Royal Society’s first publications in Newton’s day.

• Zombies: retracted papers that still get cited as if they never died.
• Struldbrugs: Berenbaum’s name for obsolete papers that cease to be cited.
• Slumbering monsters: bad ideas that should die but cannot.

Not publishing bad things like these in the first place would be desirable, but would require omniscience. Bad ideas will continue to be published. But journals have a solution: posting retractions, right? It’s a good practice in theory:

Simply publishing a retraction clearly isn’t an effective way to “kill” a paper—retraction notices aren’t cited as often as are the retracted papers themselves, even more than a decade after their retractions….  The creation of the Retraction Watch Database by the Center for Scientific Integrity has been an invaluable resource for informing authors and publishers about retractions, but not all reference managers link automatically to the database. Even once a retraction notice is found, as I discovered in writing this editorial, it’s not always clear how to cite the retraction rather than the offending article.

Citing the retraction may actually increase the citation count for the offending article, raising its status before the scientific community viewing high citation count as a measure of value. And how does one add retraction notices to papers hidden in voluminous tomes of published papers in the endless stacks of research libraries at all the world’s universities? There they will sit for unwary grad students to cite in their doctoral research.

As editor, Berenbaum cannot purge her journal of these horrors, because that would be a type of censorship reminiscent of the Memory Hole in 1984. Historians and scientists need to know about science’s mistakes, blunders and wrong turns: even the cases of fraud in the history of science so that people can learn their lessons. Karl Pearson’s racist Annals of Eugenics will remain on the shelves as a warning but could trip the unwary.

The availability of primary documents has helped two creationist writers who have published valuable historical lessons from the blunders of science:

• Jerry Bergman: Evolution’s Blunders, Frauds and Forgeries
• Randy Guliuzza, Twenty Evolutionary Blunders: Dangers & Difficulties of Darwinian Thinking

The Possibility of Babies in the Bathwater

Then there’s the problem of Sleeping Beauties in the cobweb-infested stacks. Berenbaum gives examples of good papers, long forgotten, with potential to awaken again.

Yet there is a cost of expunging papers that have apparently outlived their usefulness—doing so risks losing the chance of restoring their utility in new contexts made possible by future scientific advances. Although most papers accrue the bulk of their citations within five years of publication, others remain ignored for decades until, due to a change in the state of knowledge, often in a new field, they gain (or regain) relevance. Van Raan coined the term “Sleeping Beauty phenomenon” for this revivification, and Ke et al. created a metric, the beauty coefficient, that takes into account the number of citations received by a paper and the length of time after publication that the citations are received.

That is another reason Berebaum can’t clean up the library of science willy-nilly. She found herself profiting from time spent browsing online digital copies of Pearson’s discredited journal of eugenics, even if it was not pleasurable.

In 1955, Annals of Eugenics became Annals of Human Genetics, reflective of increasing discomfort with the status of eugenics as a scientific discipline. I could read Annals of Eugenics because Annals of Human Genetics, University College London, and Blackwell Publishing digitized its full run (1925–1954). Instead of opting to ignore its existence, this group did us all a favor by preserving this example of problematical science. It serves as a reminder that, along with Sleeping Beauties, the literature contains slumbering monsters, and part of the iterative process of scientific research is calling out and remembering the mistakes so as not to repeat them.

After that, Berenbaum read Pollyanna.

Berenbaum ends cheerfully with dreams of “calling out and remembering the mistakes so as not to repeat them.” But earlier she said that scientists are not aware of the mistakes, because retraction methods don’t work. Retracted papers are cited more often than the retraction notices! And library shelves are full of zombies and monsters haunting the Sleeping Beauties. What kind of happy dream world is she living in?

Listen up, those who tweet glibly that we must “Follow the Science.” Which science? The slumbering monsters? The zombies? The struldbrugs? Like a body ridden with metastatic cancer but still functioning in remission, science is a mess of good and bad, healthy and unhealthy, young and vibrant vs old and dying.

Here’s a danger that Berenbaum did not mention: the purging of history by censorship. In today’s Cancel Culture, one cannot guarantee that primary sources will remain available online. It would be easy for Big Government or Big Science to silently shadow-ban embarrassing publications. How many researchers would go to the trouble of using the Wayback Machine or stalking the stacks at a big library to get to the sources? What if Wayback gets sent down the Memory Hole?

Even worse would be alterations. Digital versions are easy to tweak. Devious editors can secretly alter statements in scientific literature like the hidden editors at Wikipedia do to that online source’s articles; for instance, advocates of intelligent design, and some conservative politicians, are unable to correct outright lies in articles about them or their beliefs, because hidden editors accountable only to themselves immediately switch them back. This is an abominable practice with as yet no solution. When Big Science, Big Media and Big Government start engaging in this kind of manipulation of history, no online source will be reliable.

We know by comparing Darwin’s six published editions of The Origin of Species that he made substantive changes to his theory in the years 1859 and 1872. This information is valuable to historians seeking to understand how he responded to criticisms in those years. What if institutions decide to purge the old editions and create an “official” edition of Darwin’s book? Another thought: what will happen when Darwinism falls? What will happen when climate science’s predictions fail to come to pass and the world does not come to an end? How about if the experts at the CDC are falsified for numerous bad calls about Covid-19? It will be tempting to erase the embarrassing history of dogmatic claims. The public deserves to know the blunders as well as the commendable research.

Berenbaum did us a favor by turning the lights on the dark cobweb-infested halls of scientific literature. But now she needs to put down Pollyanna and do more to warn the public about the monsters and zombies walking around. Students are still being spoon-fed a simplistic vision of “The Scientific Method” (there is no such thing applicable to every field) and false notions that science is self-correcting. No; the replication crisis is real; “the science” is a blend of nutrients and toxins; and critical thinking is mandatory to sort the good from the bad.

As we often say, science is worthless without integrity. Science is always mediated by fallible humans who are not above the laws of logic and morality. Don’t follow the science blindly. Follow the good science with critical thinking.