Harry Enten, a reporter and analyst at FiveThirtyEight, writes frequently about polling and has been integral in the site’s efforts to sort out the madness during ths election. We spoke to him about how to sort out a good poll from a bad poll, and how journalists (and even politicians) can do a better job when they talk about them.
How can our readers and listeners tell the difference between a good poll and a bad poll?
Enten: Well, I think that there are two examples. There's the nerdy example, which is why a particular poll is bad, and then there's the more straightforward answer, as how to tell whether or not a poll is bad without necessarily having all the in-depth knowledge. I'll answer both of those.
The simpler way is asking yourself if you have ever heard of the poll and/or the organization that's sponsoring the poll. For example, if you hear a poll has been conducted by ABC News or NBC News or CBS News, then that poll is probably pretty good. If you haven't heard of the particular poll -- let's say the Gravis Marketing Breitbart -- poll, then that poll probably isn't necessarily up to snuff. That doesn't mean it's a terrible poll, but it does probably mean that the quality of the poll is not quite up to the standards that we would usually like to say if we were citing one particular poll in an example.
Now, what makes a good poll versus a bad poll? I think the key thing that distinguishes a good poll from a bad poll is a good poll makes sure it polls the entire population. That is, that each person in the population has an equal chance of being selected. Because that's what's really key in polling is that we have complete coverage of the entire population, especially in elections where we see great differences between young African Americans who are much likely to be Democratic, but are much harder to poll or at least get ahold of than say older white voters, who are much more easily to get a hold of, but are much more likely to be republican.
Trump has repeatedly, especially in the last couple of weeks, criticized professional polling and instead turned to these online polls that have him winning by huge margins. Do you think that that is persuasive for people? Because these polls are pretty bad. They’re not real.
Enten: They’re not real. You know, we've tried to come up with nicknames for these online polls after the debates and you know he's winning in the drudge poll by 50, 60, 70, 80, sometimes upwards of 90 percentage points. They're not real polls. I think that they do speak to his base, you know, the people that love him. But the fact is that base is only worth perhaps upwards of 40 percent of the national electorate, and that's just not going to be good enough for him to win. It's good enough for him to get good rally sizes and make him feel good. But they're not real polls. I'm kind of disappointed in some of the people I've heard celebrating those polls on television, some of his surrogates. Because there's nothing wrong with being a Donald Trump supporter in my mind, however, there's something tremendously wrong about lying and knowing that you're lying. And when it comes to these online polls, a lot of his surrogates know better than to believe that these polls are accurate representations of what's actually going on.
How could journalists do a better job using polls meaningfully in their daily reporting?
Enten: The way that journalists could do a much better job is instead of relying on a single data point, they rely on multiple data points. I think the thing that's most frustrating to me is when a journalist or a writer uses a poll to back up a point that may confirm a few interviews, when there are a whole bunch of other polls that, in fact, don't back up that point at all. In the past week there was a poll taken in the state of Florida from Bloomberg, done by Ann Selzer — who is a great pollster, no doubt about it — it showed Donald Trump ahead. But, the vast majority of the polls in Florida have, in fact, shown Hillary Clinton ahead. So, any story that was written about that particular poll needed to use the average of polls to put that particular poll into context and I'm fearful that many stories, in fact, didn't do that and I think that's something that we can work on as a profession going forward.
Listen to this podcast on iTunes, SoundCloud or Stitcher. For more, follow Harry Enten's work on FiveThirtyEight.