May 4, 2009 | David F. Coppedge

Scientific Data Can Mislead

Some recent stories should remind scientists that data do not exist in a philosophical vacuum.  Sometimes empirical measurements can be downright misleading.

  1. Pillars of creation:  The famous Hubble photo of the Eagle Nebula’s “pillars of creation” seemed to have a straightforward explanation: nearby stars were eroding the pillars with blasts of radiation.  Scientists have been studying the photo more carefully.  Some are coming up with a radically different interpretation, reported, that turns the previous explanation upside down.  The pillars were pushed into place.  Both explanations involve radiation pressure from nearby stars, but an Irish astronomer said, “There is, as yet, no clear consensus in the literature regarding the formation of the pillars”.
  2. X-rays deceive:  Nature last week began a news article with “X-ray astronomy: When appearances are deceptive.”1  For years, astronomers have tried to explain an X-ray glow around the Milky Way.  Exotic explanations like supermassive black holes had been put forth.  Now, a new map has resolved the glow into discrete sources.  This means the X-rays are probably coming from point sources like ordinary stars.  “With blurry vision, one can imagine strange and wonderful things that later turn out to be erroneous,” Shull said.  He likened this story to the fabled canals on Mars that evaporated when spacecraft dropped by to visit.
  3. Brain chaff:  Functional MRI (fMRI) is a powerful tool for discerning detail in the brain.  Trouble is, neuroscientists have not been doing very well at sifting the wheat from the chaff in the data.  In fact, Nature reported, “Nearly half of the neuroimaging studies published in prestige journals in 2008 contain unintentionally biased data that could distort their scientific conclusions.”2 
  4. Green fluorescent protein:  A workhorse of molecular biology has been green fluorescent protein (GFP), a molecule that glows.  GFP allows scientists to follow molecular interactions in the cell by inserting the gene for GFP in biochemical pathways.  Is it really a neutral probe, though?  Nature News reported that a Russian team found by accident that GFP donates electrons to nearby molecules, potentially altering their function.  The researcher said, “It is quite unexpected that GFP can interact with external molecules and donate electrons.”

A book review in Nature last week reviewed historical examples of models and data that misled scientists.3  Andrew F. Read reviewed Pavlov’s Dogs and Schroedinger’s Cat: Scenes From the Living Laboratory by Rom Harr�.3  In this book, philosopher Rom Harr� recounted the use of animals as instruments in the history of science (think of Galvani using frog legs to test electricity).  “As a lucid illustration of the messy, chaotic and glorious professional world we scientists have always inhabited,” Read commented, “this book is great.”  Stories are amusing and shocking.  Harr� tells “one of the most gripping and succinct summaries of the Lysenko fiasco in Soviet genetics” that Read has ever read. 

Yet the book is more thought-provoking than a simple storytelling exercise.  It is organized not as a history, nor by scientific subject, but by the principles of the philosophy of science.  Chapters are devoted to organisms as detecting and measuring devices, and as tools for exploration, for testing hypotheses and for modelling reality.  This organization generates an easily digested introduction to many of the key concepts of the philosophy of science.  It demonstrates vividly that there is no single way of doing science; philosophers of science are hard pressed to describe what is going on, let alone prescribe what we ought to be doing.

The only frustration Read felt was that last phrase.  Harr� left the reader hanging: what should scientists be doing?  He didn’t take up the pressing moral questions.  He said, “Only when we have a clear idea of what has been done by whom and for what purpose can we take up the pressing moral questions that must arise,“ but then later “dodged the bullet” himself by saying, “I leave the working out of moral arguments to others more qualified than I am to reach just and ethically sound conclusions.”  Read wondered who better than Harr� would be qualified to address that question.

1.  Michael Shull, “X-ray astronomy: When appearances are deceptive,” Nature 458, 1119-1121 (30 April 2009) | doi:10.1038/4581119a.
2.  Alison Abbott, “Brain imaging skewed; Double dipping of data magnifies errors in functional MRI scans,” Nature Published online 27 April 2009 | Nature | doi:10.1038/4581087a.
3.  Andrew F. Read, “Tales of top models,” Nature 458, 1113-1114 (30 April 2009) | doi:10.1038/4581113a.

With varying degrees of justification, Pavlov, Schroedinger, Lysenko, Galvani, Mesmer and others truly believed their scientific measurements and models were uncovering truths about nature – lifting a corner of the veil to discover realities beyond common-sense experience.  Once in awhile we are reminded that science is a messy project of fallible minds utilizing fallible senses to operate fallible instruments in the “messy, chaotic and glorious professional world … scientists have always inhabited.”  If it is messy for the hard sciences using real-time observation, how much more for the shadowy pasts of biological history and cosmology?
    The history of science is a fascinating tale when told honestly (without the triumphalist March of Progress metaphor).  There were arguably more false starts and reversals than successes.  Some of the greatest successes (e.g., Newtonian gravity) proved vulnerable to further investigation centuries later.  We laugh at the alchemists spending centuries in a vain quest to turn base metals into gold, but in their own day, they were the “scientists” – the educated scholars, seeking understanding.  Surely such foibles are not happening any more today, are they?  Are they?
    Read’s cautionary advice is worth repeating.  The book he reviewed provided only an easily digested introduction to many of the “key concepts of the philosophy of science.  It demonstrates vividly that there is no single way of doing science; philosophers of science are hard pressed to describe what is going on, let alone prescribe what we ought to be doing.”  Scientists today could use more humility.  They should not stand so high and mighty above scholars in other fields.  In some ways scientists are like drunkards stumbling around, once in awhile knocking their heads against the lamp posts of reality (see First Law of Scientific Progress and Young’s Law in the right sidebar).  The quote by Daniel Robinson at the end of the 12/11/2005 commentary bears repeating.

(Visited 6 times, 1 visits today)
Categories: Cell Biology, Physics

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.