In Praise of Rejects
Today's failed study could lead to tomorrow's smart policy—if we let it.
Everyone likes to read about research that succeeds. But sometimes it can be more useful to know what fails. Nowhere is this more true than in the social sciences, which the policy community often turns to for answers on how the world works. The trouble is, for each study that sees print, a researcher might have discarded a file drawer of findings that did not support her theories. Editors and readers don't dwell on-and may never see-findings that are inconclusive, fail to confirm the researcher's hypothesis, or can't be easily explained by existing theories. These so-called "negative results" get buried because it's simply bad marketing to publish wrong answers.
Everyone likes to read about research that succeeds. But sometimes it can be more useful to know what fails. Nowhere is this more true than in the social sciences, which the policy community often turns to for answers on how the world works. The trouble is, for each study that sees print, a researcher might have discarded a file drawer of findings that did not support her theories. Editors and readers don’t dwell on-and may never see-findings that are inconclusive, fail to confirm the researcher’s hypothesis, or can’t be easily explained by existing theories. These so-called "negative results" get buried because it’s simply bad marketing to publish wrong answers.
But this is a shame, because we could learn a lot from seeing all the evidence. The Journal of Spurious Correlations, which I helped found, has received a number of conventional-wisdom-puncturing articles that couldn’t be published elsewhere because they present negative results. One article, for example, failed to find a positive correlation between the presence of women in government and a lower incidence of corruption, challenging the age-old notion that women are less corrupt than men. Another one reexamined a canonical study demonstrating that high levels of income inequality in moderately wealthy countries undermine democracy. When new data were used to test the old paradigm, the results were-compellingly-inconclusive. Neither of these studies is likely to be accepted by a mainstream journal because the answers they give are unclear. But they still have crucial information to offer, and discarding them would be a mistake.
Publishing rigorous, informative results that seem unsellable will, we hope, give them the prestige and the audience they deserve. It will help update a scientific culture that prefers the simple and conclusive to the complex and open-ended, and often misses out on valuable information as a result. And it will mean that science can move forward to new questions instead of getting snagged on the easy answers. We can’t learn from our mistakes if we don’t even know what they are.
More from Foreign Policy
At Long Last, the Foreign Service Gets the Netflix Treatment
Keri Russell gets Drexel furniture but no Senate confirmation hearing.
How Macron Is Blocking EU Strategy on Russia and China
As a strategic consensus emerges in Europe, France is in the way.
What the Bush-Obama China Memos Reveal
Newly declassified documents contain important lessons for U.S. China policy.
Russia’s Boom Business Goes Bust
Moscow’s arms exports have fallen to levels not seen since the Soviet Union’s collapse.
Sign up for World Brief
FP’s flagship evening newsletter guiding you through the most important world stories of the day. Delivered weekdays.
By submitting your email, you agree to the Privacy Policy and Terms of Use and to receive email correspondence from us. You may opt out at any time.
