You Can Trust Scientists (to Be Fallible)
With theory overturns like these, one gets to wonder about the privileged status of scientists in our culture.
False positives: For 25 years, neuroscientists have relied on functional MRI imaging (fMRI) to draw inferences about the brain. Patients have been asked questions and told to engage in mental activities while scientists watched areas light up on their brain scans. Too bad researchers didn’t know the software was buggy when they wrote up their results. A new study published on PNAS finds that possibly 70% of the research published in the last quarter century is flawed. “These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results,” they say. Science Daily reports, “Common statistical methods used to analyse brain activity through images taken with MRI scanners cannot be trusted.”
False trust: Many researchers using fMRI simply had faith in software systems applying “old, unreliable analysis methods.” A statistician from the University of Pittsburgh commented on the above study later in PNAS. Within the “sobering tale” was a bug in software that went unnoticed for 15 years. Science Daily offers “Ten simple rules to use statistics effectively.” Nice in theory, but much of the analysis is done in software these days, which scientists use without necessarily understanding or validating it.
Eat your butter: After being told for decades to avoid butter fat and high cholesterol, this week we are being told by Live Science, “Butter may not be bad for your heart.” Science Daily points to new evidence of “Little to no association between butter consumption, chronic disease or total mortality.” Skeptics may want to study the funding sources and potential conflicts of interest listed in the PLoS One paper.
Eat your cholesterol: Another Science Daily article says there is “no association between ‘bad cholesterol’ and elderly deaths,” after a study of 68,000 seniors. Low-density lipoproteins (LDL) have been considered bad for years, leading to hardening of the arteries, but is that opinion mistaken, at least for those over 60? The study “also raises questions about the benefits of statin drug treatments” often prescribed to reduce cholesterol.
“Our findings provide a contradiction to the cholesterol hypothesis,” concluded Diamond. “That hypothesis predicts that cardiovascular disease starts in middle age as a result of high LDL-C cholesterol, worsens with aging, and eventually leads to death from cardiovascular disease. We did not find that trend. If LDL-C is accumulating in arteries over a lifetime to cause heart disease, then why is it that elderly people with the highest LDL-C live the longest? Since people over the age of 60 with high LDL-C live the longest, why should we lower it?”
Dinosaur myths: Nick Longrich is proud of his debunking. On The Conversation, he lists “The top six dinosaur myths and how we busted them.” This implies that the myths misled scientists for many years, who in turn misled the public. The question remaining is whether scientists have replaced old myths with new myths, particularly his #6, that birds are the direct descendants of dinosaurs.
Clinical trial troubles: Various opinions are shared in the comments after Derek Lowe’s blog post for Science Magazine about shortcomings of clinical trials in the New England Journal of Medicine. The scientific realists take swipes at the social scientists, but is anyone convinced that published trial results are completely trustworthy? Lowe, a realist, moans, “As you can see, things get pretty philosophical, and they also get pretty political, too, with all sorts of stuff being dragged in from the social science hallways about privileged ways of knowledge, etc.”
Predatory journals: You can trust peer-reviewed journals, can’t you? Jeffrey Beall doesn’t think so. He wrote a letter to Nature complaining about “predatory journals” that are “threatening the credibility of science.” He thinks they should be banned. What are they? According to him, they publish junk science, fringe science, and activist research. He even casts aspersions on open-access journals: “Finally, advocates of open-access publication must stop pretending that the author-pays model is free of serious, long-term structural problems,” he says. “Just because it works well in a few cases doesn’t mean it always works.” But who decides when it “works” or not—when it supports the consensus? Who decides which journals are predatory, or which articles within a journal are tainted? Surely Nature has never been guilty of that, has it? Conflict of interest that they published his letter, perhaps?
The anti- antimatter matter: It’s a cosmological problem that won’t go away. ABC News posted an article about the “Antimatter Mystery” that has plagued cosmology for decades. “We shouldn’t be here; physics says so” reads one of the sections. Astronomers keep hoping some tiny asymmetry will be found in particle physics that leaves enough matter left over to tip the balance. “Theories are great, but science is built on evidence,” ABC science writer Bernie Hobbs reminds us. Speaking of things that shouldn’t exist (like our universe), PhysOrg tells about exoplanet K2-39b, “a planet that shouldn’t be there at all.” It orbits so close to its subgiant star, it should have been ripped to shreds by tidal forces, according to commonly used models. Back to the drawing board.
More on the reproducibility crisis: One of science’s greatest claims to privileged knowledge, reproducibility, has been debunked often lately. In Nature, Monya Baker argues one reason for it: “Muddled meanings hamper efforts to fix reproducibility crisis.” This crisis is so convoluted, it folds back on itself. “A semantic confusion is clouding one of the most talked-about issues in research. Scientists agree that there is a crisis in reproducibility, but they can’t agree on what ‘reproducibility’ means.” Will each discipline have to come up with its own definition and standards? At a recent meeting about the crisis, that was suggested. But if different fields each have their own standards and definitions, do they all deserve the label “science”? Maybe Finagle had it right: “Experiments must be reproducible. They should all fail the same way.”
Self-congratulation: Like any business, heads of science journals like to make themselves look good in print. Rush Holt did that for Science Magazine, praising outgoing editor Marcia McNutt and welcoming new editor-in-chief Jeremy M. Berg. But wasn’t Holt himself criticized a few years ago for his anti-Republican political bias? When AAAS editors help set policy and report on policy, who watches the watchers? Such things cannot be peer reviewed.
Too many things get lumped into the box called “science” and wrapped in shiny paper by the press. Sites like “Science Daily” and “ABC Science” present this box to the public as if it is a homogeneous assortment of gifts, all of equal value. In reality, it’s a mixed bag, part gift, part junk, and part Pandora’s Box.
All science is mediated by fallible humans. All science is tentative. Science cannot validate itself. Only minds can take a stab at validation, but they are subject to any number of competing interests: social biases, worldview preferences, conflicts of interest and more. Some biases can be minimized, but they cannot be eliminated due to the infinite and creative capacity of the mind to fool itself.
Reality is real, but real human scientists can be really fallible. Always evaluate the evidence. That takes logic, not science.