Anthropologists and Psychologists Under Fire for Flawed Methods

Poor sampling and other errors may undermine many things scientists think they know about mankind.

In a special edition, the Proceedings of the National Academy of Sciences looked into growing complaints about psychology and anthropology. Here are what some of the papers had to say:

Broadening horizons: Sample diversity and socioecological theory are essential to the future of psychological science (Michael D. Gurven, PNAS). “The present lack of sample diversity and ecological theory in psychological science fundamentally limits generalizability and obstructs scientific progress.”

Should social scientists be distanced from or engaged with the people they study? (Nzinga et al, PNAS). Anthropologists have thought they need to maintain their distance from the subjects they study. “More broadly, we argue that social and educational sciences need skepticism, interestedness, and engagement, not distancing.”

Pressing questions in the study of psychological and behavioral diversity (Hruschka et al, PNAS). This paper begins with a damning quote for the whole field of psychology. Reading this is like shock treatment:

Extreme biased sampling of research participants and the neglect of their cultural context are increasingly recognized as threats to the generalizability of much of what we know about human thought and behavior. In addition to reinforcing narrow views of what it means to be human, these parochial research practices have also shaped the methodological core of the human sciences by favoring tasks that are tailored to the skills, motivations, and social expectations of a very rarefied set of humanity. Despite decades of calls for reform, there is little evidence that increasing awareness of this threat has led to changes in practice or publishing.

Daniel Hruschka continues his diatribe on The Conversation, saying, “You can’t characterize human nature if studies overlook 85 percent of people on Earth.” How influential has this fake science been on society? He says in plain English,

Over the last century, behavioral researchers have revealed the biases and prejudices that shape how people see the world and the carrots and sticks that influence our daily actions. Their discoveries have filled psychology textbooks and inspired generations of students. They’ve also informed how businesses manage their employees, how educators develop new curricula and how political campaigns persuade and motivate voters.

But a growing body of research has raised concerns that many of these discoveries suffer from severe biases of their own. Specifically, the vast majority of what we know about human psychology and behavior comes from studies conducted with a narrow slice of humanity – college students, middle-class respondents living near universities and highly educated residents of wealthy, industrialized and democratic nations.

More than 90% of the studies that supposedly tell us about human beings come from just 15% of the people on the planet. That’s WEIRD: “Western, educated, industrialized, rich and democratic.”

Toward a psychology of Homo sapiens: Making psychological science more representative of the human population (Rad et al, PNAS). How generalizable are published conclusions when “most research published in our leading journals has relied on sampling WEIRD (Western, educated, industrialized, rich, and democratic) populations”? There’s trouble in psychology departments. “To take us forward, we offer a set of concrete proposals for authors, journal editors, and reviewers that may lead to a psychological science that is more representative of the human condition.”

An abundance of developmental anomalies and abnormalities in Pleistocene people (Erik Trinkaus, PNAS). And yet they go on; Erik Trinkaus proceeds to describe people he never met, because they were all dead long ago.

Polygenic adaptation and convergent evolution on growth and cardiac genetic pathways in African and Asian rainforest hunter-gatherers (Bergey et al, PNAS). This group treats distinguished human beings of the Pygmy description as their lab rats.

Cultural impediments to learning to cooperate: An experimental study of high- and low-caste men in rural India (Brooks et al, PNAS). This team uses caste members of India as their lab rats. “The results overturn earlier findings….” Maybe the roles should be reversed, since scientists can’t get it right. Let the caste members use the scientists as their lab rats.

Current Biology issued this paper questioning earlier studies:

Human Cooperation: The Hunter-Gatherer Puzzle (Joseph Heinrich, Current Biology). “A new study among the Hadza, one of the few surviving foraging populations, challenges popular approaches to cooperation while suggesting a central role for cultural transmission.”

Psychologists have a WEIRD understanding of humanity. Physician, heal thyself. Even beyond the WEIRD people they study, these social scientists often use rigged games to tweak responses from people in unrealistic situations. How reliable is that?

To understand people, start with the Manufacturer’s manual.