The File Drawer Effect, or Why Your High School Psychology Class was Useless

//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js

(adsbygoogle = window.adsbygoogle || []).push({
google_ad_client: “ca-pub-4606276359746099”,
enable_page_level_ads: true
});

The world of psychology was rocked recently by a study showing that huge chunks of psychology research weren’t able to be replicated. High-profile studies, like the one linking feeling clean and pure to the judgment of other people’s morals, were involved in The Great Debunking. 

This problem isn’t unique to social psychology. Most fields of science, especially for lower-profile, IFL-science-style studies, are plagued by this replication bias. Even cancer research is impacted.

There’re a few issues contributing. For one, there’s a bias toward findings that are surprising and prove something. That bias, which leads to studies with negative or confirmation results, to end up sitting around file drawers unpublished, is called “publication bias,” or “the file drawer effect.” It also discourages studies simply replicating the results of previous studies. No one makes their mark that way.

The scientific community has been making efforts to fix the problem, and it doesn’t affect high-profile issues of scientific consensus, like the Big Bang, Evolution, or Climate Change, but we have a long way to go.

Image By Sara Franses (Own work) [CC BY-SA 4.0
(http://creativecommons.org/licenses/by-sa/4.0)%5D, via Wikimedia Commons


Given the stakes involved and its centrality to the scientific method, it may seem perplexing that replication is the exception rather than the rule. The reasons why are varied, but most come down to the perverse incentives driving research. Scientific journals typically view “positive” findings that announce a novel relationship or support a theoretical claim as more interesting than “negative” findings that say that things are unrelated or that a theory is not supported. The more surprising the positive finding, the better, even though surprising findings are statistically less likely to be accurate.
http://www.slate.com/articles/health_and_science/science/2014/07/replication_controversy_in_psychology_bullying_file_drawer_effect_blog_posts.html

 

Publication bias is a type of bias occurring in published academic research. It occurs when the outcome of an experiment or research study influences the decision whether to publish (or otherwise distribute) it. Publication bias is of interest because literature reviews of claims about support for a hypothesis or values for a parameter will themselves be biased if the original literature is contaminated by publication bias.
https://en.wikipedia.org/wiki/Publication_bias

 

“Positive results in research studies overall, became 22% more likely to appear in scientific journals from 1990 to 2007,” says the study, which looked at a sample of 4,656 papers over this time period, looking for trends in science journals.
[…]
The analysis looked at studies where authors proposed a hypothesis and then sought to test it, either confirming it for a positive result, or not. Overall, 70.2% of papers were positive in 1990–1991 and 85.9% were positive in 2007. “On average, the odds or reporting a positive result have increased by around 6% every year, showing a statistically highly significant trend,” says the study.
http://content.usatoday.com/communities/sciencefair/post/2011/09/science-studies-neglecting-negative-results/1#.V-r3E_ArK70

//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js

(adsbygoogle = window.adsbygoogle || []).push({});

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s