Abstract
Following Donald Trump’s unexpected victory in the 2016 US presidential election, the American Association for Public Opinion Research announced that “the polls clearly got it wrong” and noted that talk of a “crisis in polling” was already emerging. Although the national polls ended up being accurate, surveys just weeks before the election substantially over-stated Clinton’s lead and state polls showed systematic bias in favor of Clinton. Different explanations have been offered for these results, including non-response bias and late deciders. We argue, however, that these explanations cannot fully account for Trump’s underperformance in October surveys. Utilizing data from two national polls that we conducted in October of 2016 (n>2100 total) as well as 14 state-level polls from October, we find consistent evidence for the existence of “hidden” Trump supporters who were included in the surveys but did not openly express their intention to vote for Trump. Most notably, when we account for these hidden Trump supporters in our October survey data, both national and state-level analyses foreshadow Trump’s Election Day support. These results suggest that late-breaking campaign events may have had less influence than previously thought and the findings hold important implications for how scholars, media, and campaigns analyze future election surveys.
Appendix 1: State Polls Analyzed in Figure 1
As explained in the text, the state polls analyzed in Figure 1 were selected based on the following criteria: a sample of likely voters, live telephone interviews, in the field around early October, and included vote intentions for the presidential and Senate candidate. If more than one survey in a single state met these criteria, we selected the survey conducted closest to October 1. Additional survey details appear below in Table A.1.
Details for State Surveys Analyzed in Figure 1
Appendix 2: National Polls Analyzed in Figure 2
The data in Figure 2 come from the Roper Center for Public Opinion Research iPOLL database. The survey questions reported in Figure 2 typically followed the format, “If the (2016) presidential election were being held today and the candidates were Hillary Clinton and Tim Kaine, the Democrats, Donald Trump and Mike Pence, the Republicans, Gary Johnson and Bill Weld of the Libertarian Party, and Jill Stein and Ajamu Baraka of the Green Party, for whom would you vote? Would you lean toward Clinton and Kaine, Trump and Pence, Johnson and Weld, or Stein and Baraka?” After asking this initial vote intention question, several surveys followed up with respondents who indicated a third party candidate, do not know, or no answer, by asking, “If the only candidates were Hillary Clinton and Tim Kaine the Democrats and Donald Trump and Mike Pence the Republicans, for whom would you vote?” To ensure the results in Figure 2 are as representative as possible, we include the results from both formats when they are available. Additional survey details follow.
USABC.102316.R03: ABC News. ABC News Poll, Oct, 2016 [survey question]. USABC.102316.R03. ABC News [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USABC.102516.R03: ABC News. ABC News Poll, Oct, 2016 [survey question]. USABC.102516.R03. ABC News [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USABC.102316.R05: ABC News. ABC News Poll, Oct, 2016 [survey question]. USABC.102316.R05. ABC News [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USABC.102516.R05: ABC News. ABC News Poll, Oct, 2016 [survey question]. USABC.102516.R05. ABC News [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USABC.102616.R03: ABC News. ABC News Poll, Oct, 2016 [survey question]. USABC.102616.R03. ABC News [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USABC.102616.R05: ABC News. ABC News Poll, Oct, 2016 [survey question]. USABC.102616.R05. ABC News [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USABCWP.102716.R04: ABC News/Washington Post. ABC News/Washington Post Poll, Oct, 2016 [survey question]. USABCWP.102716.R04. ABC News/Washington Post [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USABCWP.102716.R03: ABC News/Washington Post. ABC News/Washington Post Poll, Oct, 2016 [survey question]. USABCWP.102716.R03. ABC News/Washington Post [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USABCWP.101616.R05: ABC News/Washington Post. ABC News/Washington Post Poll, Oct, 2016 [survey question]. USABCWP.101616.R05. ABC News/Washington Post [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USABCWP.101616.R03: ABC News/Washington Post. ABC News/Washington Post Poll, Oct, 2016 [survey question]. USABCWP.101616.R03. ABC News/Washington Post [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USARG.102116A.R01: American Research Group. American Research Group Poll, Oct, 2016 [survey question]. USARG.102116A.R01. American Research Group [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USAP.102616G.R05: Associated Press. Associated Press/GfK Knowledge Networks Poll, Oct, 2016 [survey question]. USAP.102616G.R05. GfK Knowledge Networks [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USPRRI.101916.R03: Brookings Institution. PRRI/Brookings Survey, Oct, 2016 [survey question]. USPRRI.101916.R03. PRRI [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USORC.20161023.Q05: Cable News Network. CNN/ORC International Poll, Oct, 2016 [survey question]. USORC.20161023.Q05. ORC International [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USORC.102416.R01A: Cable News Network. CNN/ORC International Poll, Oct, 2016 [survey question]. USORC.102416.R01A. ORC International [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USCBS.101716.R04: CBS News. CBS News Poll, Oct, 2016 [survey question]. USCBS.101716.R04. CBS News [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USCBS.101716.R07: CBS News. CBS News Poll, Oct, 2016 [survey question]. USCBS.101716.R07. CBS News [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USPSRA.102716K.RV03: Henry J. Kaiser Family Foundation. Kaiser Health Tracking Poll, Oct, 2016 [survey question]. USPSRA.102716K.RV03. Princeton Survey Research Associates International [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USPSRA.102716K.RV02: Henry J. Kaiser Family Foundation. Kaiser Health Tracking Poll, Oct, 2016 [survey question]. USPSRA.102716K.RV02. Princeton Survey Research Associates International [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USPSRA.102716.R011A: Pew Research Center for the People & the Press. Pew Research Center for the People & the Press Political Survey, Oct, 2016 [survey question]. USPSRA.102716.R011A. Princeton Survey Research Associates International, Abt SRBI [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USPSRA.102716.R010A: Pew Research Center for the People & the Press. Pew Research Center for the People & the Press Political Survey, Oct, 2016 [survey question]. USPSRA.102716.R010A. Princeton Survey Research Associates International, Abt SRBI [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USQUINN.101916.R01: Quinnipiac University Polling Institute. Quinnipiac University Poll, Oct, 2016 [survey question]. USQUINN.101916.R01. Quinnipiac University Polling Institute [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USQUINN.100716.R02: Quinnipiac University Polling Institute. Quinnipiac University Poll, Oct, 2016 [survey question]. USQUINN.100716.R02. Quinnipiac University Polling Institute [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USQUINN.100716.R01: Quinnipiac University Polling Institute. Quinnipiac University Poll, Oct, 2016 [survey question]. USQUINN.100716.R01. Quinnipiac University Polling Institute [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
USPRRI.101116.R04: The Atlantic. PRRI/The Atlantic Survey, Oct, 2016 [survey question]. USPRRI.101116.R04. PRRI [producer]. Cornell University, Ithaca, NY: Roper Center for Public Opinion Research, iPOLL [distributor], accessed Jun-13-2017.
Appendix 3: MRP Results for All 11 Swing States
Figure 4 in the text reported our MRP estimates and the actual vote share for the seven swing states with 10 or more electoral college votes. Figure A1 presents the results for all 11 swing states. Not surprisingly, our MRP estimates are much less accurate with the four states with fewer than 10 electoral college votes (CO, IA, NH, and NV). These states, after all, have smaller populations and are the least well represented in our surveys. Nevertheless, the key result holds. Unlike state surveys, which consistently underestimated Trump support, even when we consider all 11 swing states, our estimates do not reflect “correlated errors.” Our estimates (hollow dots) over-state Trump’s actual vote share (solid dots) in five states and underestimate Trump support in six states. Although more data would certainly improve our estimates, by employing MRP with one national-level survey from October, we do not systematically underestimate Trump support.
Trump support in October based on our GfK poll (solid dots) and actual vote support (hollow diamonds) for the 11 battleground states. Horizontal bars reflect 95% uncertainty estimates. For NC, the MRP estimate and actual vote overlap obscuring the hollow diamond.
References
American Association for Public Opinion Research (AAPOR) (2017) “An Evaluation of 2016 Election Polls in the United States.” http://www.aapor.org/getattachment/Education-Resources/Reports/AAPOR-2016-Election-Polling-Report.pdf.aspx.Search in Google Scholar
Arcuri, Luciano, Luigi Castelli, Silvia Galdi, Cristina Zogmaister and Alessandro Amadori (2008) “Predicting the Vote: Implicit Attitudes as Predictors of the Future Behavior of Decided and Undecided Voters,” Political Psychology, 29(3):369–387.10.1111/j.1467-9221.2008.00635.xSearch in Google Scholar
Bahadur, Nina (2015) “18 Real Things Donald Trump Has Actually Said About Women,” Huffington Post, August 19, sec. Women. http://www.huffingtonpost.com/entry/18-real-things-donald-trump-has-said-about-women_us_55d356a8e4b07addcb442023.Search in Google Scholar
Blake, Aaron (2016) “How America Decided, at the Last Moment, to Elect Donald Trump,” The Washington Post. Nov. 17, 2016.Search in Google Scholar
Carmon, Irin (2016) “Trump’s Worst Offense? Mocking Disabled Reporter, Poll Finds,” NBC News. August 11. Accessed January 4. http://www.nbcnews.com/politics/2016-election/trump-s-worst-offense-mocking-disabled-reporter-poll-finds-n627736.Search in Google Scholar
Connors, Elizabeth, Samara Klar and Yanna Krupnikov (2016) “There May Have Been Shy Trump Supporters After All,” November 12. The Washington Post. https://www.washingtonpost.com/news/monkey-cage/wp/2016/11/12/there-may-have-been-shy-trump-supporters-after-all/?utm_term=.ad6f7a32adbc.Search in Google Scholar
DeFour, Matthew (2016) “Donald Trump Wins Presidency after Stunning Victory in Wisconsin,” Associated Press. November 8. http://interactives.ap.org/2016/general-election/.Search in Google Scholar
Dropp, Kyle (2016) “How We Conducted Our ‘Shy Trumper’ Study” Morning Consult. https://morningconsult.com/2016/11/03/shy-trump-social-desirability-undercover-voter-study/.Search in Google Scholar
Enns, Peter K. and Julianna Koch (2013) “Public Opinion in the US States 1956 to 2010,” State Politics & Policy Quarterly, 13(3):349–372.10.1177/1532440013496439Search in Google Scholar
Enns, Peter K. and Brian Richman. 2013. “Presidential Campaigns and the Fundamentals Reconsidered,” The Journal of Politics, 75(3):803–820. doi:10.1017/S0022381613000522.10.1017/S0022381613000522Search in Google Scholar
Enns, Peter K. and Jonathon P. Schuldt. 2016. “Are There Really Hidden Trump Voters?” The New York Times, November 7. http://www.nytimes.com/2016/11/07/opinion/are-there-really-hidden-trump-voters.html?_r=0.Search in Google Scholar
Enten, Harry. 2016. “Shy’ Voters Probably Aren’t Why The Polls Missed Trump,” FiveThirtyEight. https://fivethirtyeight.com/features/shy-voters-probably-arent-why-the-polls-missed-trump/.Search in Google Scholar
Erikson, Robert S. and Christopher Wlezien (2012) The Timeline of Presidential Elections: How Campaigns Do (and Do Not) Matter. Chicago: University of Chicago Press.10.7208/chicago/9780226922164.001.0001Search in Google Scholar
Franklin, Charles (2016) “The Polls Are Not Broken. Say It Again: The Polls Are Not Broken,” Washington Post. Accessed January 3. https://www.washingtonpost.com/news/monkey-cage/wp/2016/06/02/the-polls-are-not-broken-say-it-again-the-polls-are-not-broken/?utm_term=.453bda7f8c17.Search in Google Scholar
Galdi, Silvia, Luciano Arcuri and Bertram Gawronski (2008) “Automatic Mental Associations Predict Future Choices of Undecided Decision-Makers,” Science, 321(5892):1100–1102.10.1126/science.1160769Search in Google Scholar
Gelman, Andrew and Thomas C. Little (1997) “Poststratification into Many Categories Using Hierarchical Logistic Regression,” Survey Methodology, 23(2):127–135.Search in Google Scholar
Graham, David A. (2016) “Trump Brags About Groping Women,” The Atlantic, October 7. http://www.theatlantic.com/politics/archive/2016/10/the-trump-tapes/503417/.Search in Google Scholar
Groves, Robert M. (2006) “Nonresponse Rates and Nonresponse Bias in Household Surveys,” Public Opinion Quarterly 70(5):646–675.10.1093/poq/nfl033Search in Google Scholar
Hillygus, Sunshine D. (2011) “The Evolution of Election Polling in the United States,” Public Opinion Quarterly, 75(5):962–981.10.1093/poq/nfr054Search in Google Scholar
Jacobs, Jennifer and Billy House (2016) “Trump Says He Expected to Lose Election Because of Poll Results,” Boomberg Politics, https://www.bloomberg.com/politics/articles/2016-12-14/trump-says-he-expected-to-lose-election-because-of-poll-results.Search in Google Scholar
Katz, Josh (2016) “2016 Election Forecast: Who Will Be President?” The New York Times, July 19. http://www.nytimes.com/interactive/2016/upshot/presidential-polls-forecast.html.Search in Google Scholar
Klar, Samara and Yanna Krupnikov (2016) Independent Politics: How American Disdain for Parties Leads to Political Inaction. New York: Cambridge University Press.10.1017/CBO9781316471050Search in Google Scholar
Kohn, Nate, Josh Katz and Kevin Quealy (2016) “Putting the Polling Miss of the 2016 Election in Pespective,” New York Times. https://www.nytimes.com/interactive/2016/11/13/upshot/putting-the-polling-miss-of-2016-in-perspective.html.Search in Google Scholar
Kreuter, Frauke, Stanley Presser and Roger Tourangeau (2008) “Social Desirability Bias in CATI, IVR, and Web Surveys,” Public Opinion Quarterly, 72(5):847–865.10.1093/poq/nfn063Search in Google Scholar
Langer, Gary (2016) “Shift in the Electorate’s Makeup Tightens the Presidential Contest,” ABC News. http://abcnews.go.com/Politics/shift-electorates-makeup-tightens-presidential-contest-poll/story?id=43142198.Search in Google Scholar
Lax, Jeffrey R. and Justin Phillips (2009) “How Should We Estimate Public Opinion in the States?” American Journal of Political Science, 53(1):107–121.10.1111/j.1540-5907.2008.00360.xSearch in Google Scholar
Leip, David (2016) “Dave Leip’s Atlas of U.S. Presidential Elections,” http://uselectionatlas.org/.Search in Google Scholar
Linzer, Drew (2016) “The forecasts were wrong. Trump won. What happened?” Votamatic. Nov. 16. http://votamatic.org/the-forecasts-were-wrong-trump-won-what-happened/.Search in Google Scholar
Mercer, Andrew, Claudia Deane and Kyley McGeeney (2016) “Why 2016 Election Polls Missed Their Mark,” Pew Research Center.Search in Google Scholar
Moreno, Carolina (2015) “9 Outrageous Things Donald Trump Has Said About Latinos,” Huffington Post, August 31, sec. Latino Voices. http://www.huffingtonpost.com/entry/9-outrageous-things-donald-trump-has-said-about-latinos_us_55e483a1e4b0c818f618904b.Search in Google Scholar
Nederhof, Anton J. (1985) “Methods of Coping with Social Desirability Bias: A Review,” European Journal of Social Psychology, 15(3):263–280.10.1002/ejsp.2420150303Search in Google Scholar
O’Connor, Lydia and Daniel Marans (2016) “Here Are 13 Examples Of Donald Trump Being Racist,” Huffington Post, February 29. http://www.huffingtonpost.com/entry/donald-trump-racist-examples_us_56d47177e4b03260bf777e83.Search in Google Scholar
Pew Research Center. 2012. “Assessing the Representativeness of Public Opinion Surveys,” Pew Research Center for the People and the Press. May 15. http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/.Search in Google Scholar
Politico (2016) “The Battleground Project: 2016 Swing State Map, List, Polls & News,” Election Hub. http://www.politico.com/2016-election/swing-states.Search in Google Scholar
Shepard, Stephen (2017) “Democrats Burned by Polling Blind Spot,” Politico. http://www.politico.com/story/2017/03/democrats-trump-polling-236560.Search in Google Scholar
Shirani-Mehr, Houshmand, David Rothschild, Sharad Goel and Andre Gelman (2016) “Disentangling Bias and Variance in Election Polls,” Unpublished Manuscript. https://5harad.com/papers/polling-errors.pdf.10.1080/01621459.2018.1448823Search in Google Scholar
Silver, Nate (2016a) “Election Update: Where Are The Undecided Voters?” FiveThirtyEight. October 25. http://fivethirtyeight.com/features/election-update-where-are-the-undecided-voters/.Search in Google Scholar
Silver, Nate (2016b) “Pollsters Probably Didn’t Talk To Enough White Voters Without College Degrees,” FiveThirtyEight. December 1. http://fivethirtyeight.com/features/pollsters-probably-didnt-talk-to-enough-white-voters-without-college-degrees/.Search in Google Scholar
Taylor, Shelley E. and Susan T. Fiske (1978) “Salience, Attention, and Attribution: Top of the Head Phenomena,” In: (Berkowitz, L., ed.) Advances in Experimental Social Psychology, Vol. 11, New York: Academic Press, pp. 250–288.10.1016/S0065-2601(08)60009-XSearch in Google Scholar
Trende, Sean (2016) “It Wasn’t the Polls That Missed, It Was the Pundits,” Real Clear Politics. http://www.realclearpolitics.com/articles/2016/11/12/it_wasnt_the_polls_that_missed_it_was_the_pundits_132333.html.Search in Google Scholar
Wang, Sam (2016) “Google-Wide Association Studies,” Princeton Election Consortium. http://election.princeton.edu/2016/04/26/google-wide-association-studies/.Search in Google Scholar
Wang, Sam and Joshua Gold (2008) “You’re Undecided Now,” New York Times. https://mobile.nytimes.com/2008/10/31/opinion/31iht-edwang.1.17418207.html.Search in Google Scholar
Zaller, John R. (1992) The Nature and Origins of Mass Opinion. New York: Cambridge University Press.10.1017/CBO9780511818691Search in Google Scholar
Article note:
A previous version of this paper was presented at the 2017 Annual Conference of the American Association for Public Opinion Research in New Orleans, LA. We would like to thank Claudia Deane, Gary Langer, Sam Wang, Kathleen Weldon, and Chris Wlezien for helpful comments. We also thank GfK and Cornell’s Survey Research Institute for their support and Alex Rauter for his contribution to the surveys.
©2017 Walter de Gruyter GmbH, Berlin/Boston
