Primary Polling in 2022

Dear Readers,

This month’s newsletter discusses some of the limitations of primary polling. We review presidential primary and caucus polling in the United States since 2000 to identify patterns that should be remembered when reading the many polls that will be produced in the lead up to the May 2022 primary.

Speaking of primary polls, we will release our next Franklin & Marshall College Poll on Thursday, April 14th.

Thank you for reading,

Berwood Yost

What primary polls can, and can’t, tell us.

There is unprecedented interest in Pennsylvania’s upcoming primaries among candidates, evidenced by the number running for statewide office, and among the journalists trying to make sense of it all. This means that any and all polling data, when it appears, becomes a central talking point—either to be elevated or eviscerated depending on the reader’s particular interest. This increased interest seems to make now the right time to think about what the polls can, and can’t, tell us about these primary races.

Many people may recall the high profile primaries of the 2016 election cycle in Pennsylvania. Donald Trump and Hillary Clinton were both expected to easily win their respective primary as the polling averages showed both candidates with comfortable, double-digit leads. At the same time, the individual polls producing those averages were remarkably scattered.

Polls in the last month showed a Trump lead ranging from eight to 26 points (he won by 35), and a Clinton lead ranging from six to 28 points (she won by 12). In the end, the polls overestimated Clinton’s margin of victory and underestimated Trump’s margin.

A longer-term perspective on primary polling that includes polls from multiple states confirms that the patterns we noted for the state’s presidential primaries in 2016 are consistent with general patterns seen this century in primary polling itself.[i]

Figure 1 suggests at least three important patterns for primary polls:

• Primary polls tend to be more accurate the closer it is to Election Day.
• There is a lot of variability in primary polling estimates. Even though primary polls are more accurate as the election nears, in every primary there were a considerable number of polls conducted in the last week of the election that missed the final result by double-digit margins.
• Primary polls most often tend to underestimate the lead of the winning candidate (see table and image in the appendix).

We also learn more about primary polling when we compare their performance to the performance of general election polls. Figure 2 shows the misses of the primary election and general election polls conducted each presidential cycle since 2000. The comparison of the poll results shows that general election poll results tend to be more stable, less scattered, and closer to the final outcome than primary election poll results.

Despite the many concerns expressed about general election polls in recent years, it is clear that they are better indicators of final election outcomes than are primary polls.

Why are primary polls less accurate?

There are three main reasons that primary surveys are less accurate, and more difficult to conduct, than general election surveys. First, voters do not pay much attention to primary elections, so they are more likely to be influenced by election reporting, new information, and notable events--big events and perceptions of momentum can swing voters from one candidate to another. Swings in polling margins are one result of this inattention.

Second, uncertainty about candidate preferences happens in closed primary states like Pennsylvania because people are voting for candidates who represent their chosen party. Primary candidates’ issue positions are much more similar than they are when candidates from opposing parties face each other in a general election. This increases voter uncertainty about who they will vote for and also makes their preferences less stable and less certain.

Third, voters in primaries are less certain about whether they will actually vote. A vital task in getting an accurate picture of candidate support is determining who will vote, but primary elections make it harder to figure this out because far fewer people vote in primary than general elections.

Primary turnout in even years since 2000 has exceeded 40 percent only once among Republicans (2016) and only three times among Democrats (2008, 2016, 2020) in Pennsylvania . Compare this to the general elections during this same period where turnout has never been less than 40 percent of all registered voters.

Not surprisingly, pollsters working in a primary election have a harder time telling who will or will not vote. And this point is essential—to get accurate results pollsters must talk to those who will vote and identify those who will not.

Voter uncertainty is of course not the only thing that matters—technical choices in how voters are sampled and interviewed, how questions are designed, how “likely voters” are estimated, and how the results are interpreted also contribute to some of the differences we see in polling.

But polling can only produce accurate results when the voters themselves are sure about their intentions. The truth is that in primary elections voters take much longer to make up their minds and wait until much closer to Election Day to do so. Primary election polling simply reflects voters’ uncertainty.

Given this fact, it seems reasonable to consider more than simply which candidate is in the lead to understand an election. Undoubtedly, election outcomes are the product of how voters evaluate electoral circumstances as well as candidate qualities. The unique context of every election cycle can serve as an equally important tool for understanding what is happening, and why. Regardless of how closely the polls capture final election margins, they invariably tell us important stories about the circumstances that are driving voters’ preferences. It would be a shame to ignore that context.

Appendix

References & Resources

[i] Predictive Accuracy of the 2008 Pennsylvania Primary Pollsters

This discussion uses presidential primary polls because there is much less comparable data available for senate and gubernatorial races. The patterns identified here for national polls are consistent with an analysis of the primary polls conducted in Pennsylvania during the 2008 presidential primary.