Don’t Blame the Polls

Just like after the 2016 U.S. presidential election, American pollsters are under attack. Back then, they failed to predict the victory of Donald Trump. This time around, instead, Joe Biden is on course to become president with a much smaller margin of victory than originally expected. Immediately before Election Day, he was leading nationwide polls by more than 7 percentage points over incumbent President Trump, a lead that was almost three times higher the one enjoyed by Democratic Party contender Hillary Clinton four years ago. And in the final rush of polling, Biden also managed to surpass Trump in Florida, Georgia, and Ohio—all states that, in the end, appeared to have voted for the Republican candidate.

But polls are two-sided games. At one end, there are the pollsters, who ask the questions and are responsible for designing their surveys in a methodologically sound way. At the other, there are the interviewees, who are expected to answer faithfully. Obviously, biased responses in polls skew the data, leading to inaccurate predictions. And it is not unusual for psychology to trump statistics.

Since 2016, pollsters have worked tirelessly to improve the quality of the underlying methodology of their surveys. In particular, they tried to address the sampling bias that four years ago led to underestimates for turnout among specific demographic groups (white voters in particular) who were decisive for Trump’s victory. Websites like FiveThirtyEight also aggregate polls from different sources by assigning a rating to each of them, based on accuracy scores that are adjusted for the poll’s sample size, the performance of other polls surveying the same race, statistical biases, and other factors.

Of course, even the most statistically meticulous poll will always provide a partial view of reality. Each choice concerning its structure is subjective and discretionary. In the end, its accuracy will always lie within a statistical interval of confidence—that is, a specific margin of error, up or down.

And this interval might become especially wide if those interviewed lie or do not respond at all. While nobody can confidently say anything about those who might give fake responses, we know that this year undecided voters amounted to around 5 percent of Americans ahead of the vote. That’s a sufficient number of people to alter the outcome of the election. Some may have been genuinely unsure of whether to vote for Trump or Biden. But others, particularly moderate Republicans in affluent areas, might have simply not wanted to state their preference for a polarizing figure like Trump. In statistics, this is the so-called social desirability bias—the desire to appear to do what is perceived as the socially correct thing to do. For similar reasons, polls might have failed to fully capture the shift of Hispanic voters toward Trump in some key states.

And that is an important lesson: Even as technology and big data keep improving the quality of the polling on the pollsters’ side, at the other end the respondents will always be human beings, with their own psychologies, biases, and emotions. If we forget it, we will set ourselves up for new electoral surprises every time.