Scientific Method Evolves
There is no scientific method. There are customary procedures that depend on the field; these are subject to mutation and artificial selection.
Good-bye, P-value: A long-trusted measure of significance has come under fire. Read about it on The Conversation, where Lewis Halsey suggests it is time to let go of “one of science’s most fundamental measures“. The P-value (probability value) has been widely used in many sciences for decades, but particularly in medicine, where it is used to assess the benefit of a drug. Anything with a measured effectiveness over 5% is said to be “statistically significant” over the control or the null hypothesis.
Scientists like numbers, because they can be compared with other numbers. And often these comparisons are made with statistical analyses, to formalise the process. The broad idea behind all statistical analyses is that they allow the researcher to make somewhat objective assessments of the results of their experiments.
Who set that threshold? A fallible human being—R.A. Fisher, namely. The trouble is, researchers can fudge results (i.e., by trying the test over and over until .05 occurs by chance), to reach that arbitrary value so as to justify their results as significant. “P values are fickle friends,” Halsey says, arguing that it does not indicate reliable results but only grants an illusion of objectivity. Nature reports that a psychology journal, Basic and Applied Social Psychology, has banned the P value in its publications. According to the editors, P-value has become a crutch: “We believe that the p < .05 bar is too easy to pass and sometimes serves as an excuse for lower quality research.” Some are worried that this will make scientific research look less objective. Subjectivity is a bad word in science.
Good-bye simplicity and elegance: There’s been a teapot tempest over at Nature about scientific justification. George Ellis and Joe Silk took Richard Dawid to task about implying that simplicity and elegance were good enough to justify a scientific theory; Dawid replied this week that he never said that. Ellis and Silk were upset about theories that were lacking in empirical verifiability (see Evolution News & Views). Dawid lists the ways that empirical justification must undergird “non-empirical confirmation,” such as elegance or simplicity. But nature (the real nature, not the journal) owes no one an obligation to be simple or elegant, though it often is. On the other side of the coin, what empiricism refers to is a vexed question in philosophy of science.
Science blockers: Scientific progress is not always positive; it can be negative, putting understanding in reverse. John Brockman has edited a new book named This Idea Must Die: Scientific theories that are blocking progress. It’s a collection of essays written in response to his annual Edge question last year, “which bits of science do we want to bury?” In a review on New Scientist, Simon Ings wonders which ones on Brockman’s list should be retired: brain plasticity? Godlessness? Malthusian notions? He quotes Planck’s positivist-damaging quip: “A new scientific truth does not triumph by convincing its opponents…” he wrote, “but rather because its opponents eventually die.” Here’s an intellectual puzzle: “Some ideas cited in the book are so annoying that we would be better off without them, even though they are true.” And here’s what Ings thinks of the book: “This is a book to argue with – even to throw against the wall at times…. Every reader will have a favourite.”
Unreliable scientific method: Here are some reversals and scandals that have been in the science news recently. Findings from the 1970s that downplayed the effect of sugar on pediatric dental care was influenced by the sugar industry, Science Daily says. PhysOrg says that the “exercise hormone” irisin touted as a way to get the benefits of exercise from a pill now appears to be a myth. Is salt good or bad for you? Nobody seems to know what to think. On the one hand, it affects organs even in the absence of high blood pressure (Science Daily). But it also increases physical performance in resistance competitions (Medical Xpress) and a high-salt diet repels invading microbes (Medical Xpress). In cell biology, understanding of a cell enzyme has been flipped on its head (Medical Xpress). Finally (and these are just recent samples), National Geographic reported discovery of a dusty galaxy that shouldn’t exist. In a subsection “What Astronomers Think They Know,” Michael D. Lemonick explains that, according to theory, dust can only be formed in supernova explosions—but there wasn’t time for so much dust to accumulate in this galaxy. Astronomers were “taken aback” by what they saw.
Evolving peer review: Nature is changing its peer review policy this month. It is offering “double-blind peer review” after numerous criticisms that traditional peer review keeps some good papers out, lets poor papers in, and can punish rivals. But double-blind peer review has problems, too. In fact, there’s no one right way to peer review a scientific paper:
Alternatives to the conventional peer-review process are often proposed. Some have suggested fully open reviews, in which the names of both authors and reviewers are known. Proponents of open peer review see its transparency as a way to encourage more civil and thoughtful reviewer comments — although others are concerned that it promotes a less critical attitude.
By contrast, advocates of double-blind peer review suggest that it eliminates personal biases, such as those based on gender, seniority, reputation and affiliation.
Both systems are already in use across scholarly publishing, but there is no consensus on which is best.
Would a consensus even be capable of knowing what is best? There are tradeoffs in any method adopted. It should be obvious that no system of peer review will work without those involved being scrupulously honest. At PNAS, there’s a debate about whether science journals should prevent mistakes or publish corrections after they are published.
Political science: There should be a distinction between “science” per se, and “scientific institutions” or “the scientific community” (i.e., Big Science), with its hands dirty with politics. Nature recently wrote about the mistrust and meddling that is unsettling the US National Science Foundation. With Republicans in charge of both houses of Congress, the political landscape has changed, and the NSF must adapt, the new director thinks. Boer Deng writes, “Republicans in the US Congress have put the agency under the microscope, questioning its decisions on individual grants and the purpose of entire fields of study.” Nature thinks this has reduced morale among scientists, but lots of people in the country have low morale for various reasons, many because of the economy. There’s never enough money to do everything Big Science wants. And why is American science policy any business of the Brits?
The P-Value of Ockham’s Razor: In his latest book Being as Communion, William Dembski makes the following footnote about the principle, made famous by William of Ockham, that simplicity in explanation should override plurality: “But note: the principle of parsimony is readily abused. No doubt, explanation should avoid invoking unnecessary or superfluous entities. But it must also be true to the thing it attempts to explain.” He quotes Einstein on this, along with the summary statement “Everything should be made as simple as possible, but not simpler.” Then he blames materialism as too reductionist: “Insofar as explanation fails to account for some salient fact, it is incomplete and its parsimony can no longer rightly be regarded as an asset. Indeed, the key failing of materialism, as we shall see, is that its parsimony is purchased at the cost of misrepresenting reality.” (pp. 17-18, footnote 2).
Philosophers of science know that there is no one “scientific method,” nor are there any criteria to demarcate science from pseudoscience. Many science reporters (and cosmologists like George Ellis and Joe Silk) seem oblivious to that fact. C. S. Lewis even said there is no such thing as “scientific” thinking, only “logical” thinking. You cannot blindly crank a “scientific method” machine and crank out knowledge. There are inevitable assumptions about the nature of reality involved. There are numerous “unknown unknowns” involved. Great scientists often succeed through tacit knowledge (abductive inference), not just mechanical methods. And any method or procedure is useless without integrity. Science is distinct over other disciplines only in its subject matter (“nature,” whatever that refers to), not its need for honesty, discipline, and critique. Historians need these attributes. Theologians need these attributes. Everyone needs these attributes. We would do well to stop treating “science” as a sacred cow superior to other endeavors.
Some try to exalt science because of its results: space flight, lasers, and the works. But it should be noted that ancient peoples achieved monumental results for their time—some of which cannot be duplicated today—without following a “scientific method.” Certain methods dubbed “scientific” are merely more refined ways to organize one’s thoughts so as not to be misled, and to build on accumulated knowledge from others. Any field—law, politics, journalism, art—can progress by doing similar things. Science has its share of false leads, backtracks, and misconceptions. Some of the worst are going on right now: e.g., materialism, Darwinism, and the idea that “consensus” trumps critical thinking.
Recommended resource: Being As Communion is an interesting book that makes the case for information—not matter—constituting the ground of ontology (the most fundamental aspect of reality). It is Dembski’s third book in a trilogy, after The Design Inference and No Free Lunch, making a rigorous case for intelligent design in both philosophy and mathematics. These are good books for people who like to think deeply about the nature of reality or understand the rigorous foundation for a theistic worldview.