No tags yet.
Great explanation of "Coverage error", "Partisan non-response bias" & "Expressive responding".
A new ABC News/Washington Post poll reports that former President Donald Trump has a 51% to 42% lead over President Joe Biden.
Any one poll is subject to error. That's why we have an average.
ByG. Elliott Morris
September 26, 2023, 3:33 PM
Two new polls of the 2024 general election released this weekend showed different numbers. One poll, conducted jointly by Hart Research Associates (a Democratic pollster) and Public Opinion Strategies (a Republican firm) on behalf of NBC News, found President Biden and former President Donald Trump tied at 46 percent among registered voters. But a different survey, from ABC News and The Washington Post, found Trump up by 9 percentage points (51 percent to 42 percent) among adults. (For full disclosure: 538 is a political data journalism vertical of ABC News and has partnered with both ABC News and The Washington Post on other national polls.)
The latter poll is at odds not only with the NBC News survey, but also a simple average I calculated of all 2024 general-election polls. (Controlling for recency and sample size, that average on Saturday — before the two polls were released — had Biden leading Trump by roughly 2 points.) In its writeup of the poll, The Washington Post noted, “The difference between this poll and others, as well as the unusual makeup of Trump’s and Biden’s coalitions in this survey, suggest it is probably an outlier.”
Indeed, even good pollsters produce outlier results from time to time. But how does this happen? How can you tell when statistical error may be affecting the result of a poll? And how should we be interpreting and using these outliers, if at all? Let me share with you how I go about answering these questions.
The anatomy of an outlier poll
In order to understand how one poll can differ so much from others, you first have to understand how polls are conducted. For simplicity’s sake, we’ll limit this explanation to surveys conducted over the phone, as both the ABC News/Washington Post and NBC News polls were.
The process of polling begins with pollsters obtaining a list of phone numbers of potential interviewees. They can do that in a couple of different ways. Some pollsters use a method called random digit dialing (RDD) in which they use a computer to randomly generate a list of landline and cell phone numbers to call. Others use something called registration-based sampling (RBS), in which pollsters obtain phone numbers from voter registration records published by each state. Already, this introduces the first opportunity for error in a poll, called “coverage error.” ....