A political piece analysing the successes and failures of polling and betting companies in the 2024 US Election.

Donald Trump is more popular than ever. In what was dubbed ‘One of the closest presidential contests in recent history’ (Financial Times, 2024), Trump defied expectations, winning a sizeable majority. The Republican party won every swing state in the presidential contest, as well as winning the Senate, and the House of Representatives. The size of this victory was almost entirely unforeseen, both by the media and reputable pollsters. After a similar failure in 2016 (BBC News, 2016), and to some extent in 2020 (BBC News, 2020), the recent election represents the third major polling failure in a row. I will use the following article to evaluate the successes and failures of the polls in the election, as well as highlight essential reforms to prevent the same polling issues happening again.

Firstly, it is important to acknowledge that there is some debate over whether the polls completely failed in this election. Whilst all major polls struggled to pick a winner, some polls did pick up on the likelihood of a sizeable majority to whoever would win. To demonstrate this, I will use pollster Nate Silver. Silver became prominent in American media after his 2016 predictions gave Trump a 28.6% chance of victory, (FiveThirtyEight, 2016) more than mainstream media predictions at the time. In the day before this year’s election, Silver tweeted that after running 80,000 simulations Harris won 40,012 (2024). Silver’s tweet indicates the closeness of the election, and on the surface suggests that he believes the election will be very tight.

Closer analysis of Silver’s work paints a different picture. In fact, two weeks before the election Silver tweeted the following table, which shows that the most likely scenario (with a 24.7% probability of happening) is Trump winning every single swing state and Harris ending up with 226 electoral votes. Silver correctly predicted the election. 

Figure 1. The most likely results in the US election. (Silver, N., 2024).

Despite Silver’s success, many media outlets and experienced political commentators failed to come close to predicting the election. The Economist forecasted that Trump would be highly unlikely to win more than 48.1% of the popular vote, and it would be highly improbable for Trump to get a higher share of the popular vote than Harris (The Economist, 2024). Both events happened. The New York Times’ swing state analysis predicted Harris would win Michigan and Wisconsin, as well as having potential to win other states with ‘even’ chances (New York Times, 2024). Neither happened. Even political commentators were led astray. ‘Kamala Harris will win comfortably’, Rory Stewart tweeted (2024). It turned out not to be. So how did such reputable publications and commentators completely fail to predict the result of the election?

According to James Johnson, co-founder of JL Partners, most pollsters ‘still use single-mode polls, whether by phone or online only’ (The Times, 2024). The fact that single-mode polls are used is likely to cause bias itself. Telephone polls tend to lend themselves to older people (BBC News, 2024), which means the result is more likely to have a Republican influence (Statista, 2024). In recent years however, pollsters have increasingly transitioned to online polling (Pew Research Center, 2019). These are cheaper, more convenient both for the pollster and the pollee, and produce a larger quantity of results which should theoretically improve accuracy. In practice however, these polls are more likely to be completed by people who are young, highly-engaged, and working from home. These characteristics all increase the probability of someone voting Democrat, Johnson believes.

The shift from telephone to online polls, and the characteristics of respective pollee’s, are likely to be a major reason why polls are getting worse over time instead of getting better. The following graph shows the extent to which polls have got worse, using data from the FiveThirtyEight poll tracker (2016) (2020) (2024) and comparing it to the actual election popular vote results.

Figure 2. US Election Polling through time. (Author’s own, made using data from FiveThirtyEight).

In all three election years, the polls overestimated the Democrat vote and underestimated the Republican vote. In 2016 the inaccuracy was only 1.5%. Whilst the polls were wildly incorrect in terms of predicting the outcome of the election this was generally because of the failure to identify Trump polling well in the swing states, and its implications on the Electoral College (FiveThirtyEight, 2017). The raw data itself was not massively incorrect (The Harvard Gazette, 2017). In 2020, a year where the pandemic meant in-person polling was at an all-time low and online polling took off, the disparity was much greater. The polls were wrong by 3.9%. By 2024 in-person polling picked up slightly, however there were still far more online polls taking place than any pre-COVID year. Consequently, the disparity fell slightly to 3.3%. Nevertheless; it is clear from the data provided that in order for polling to improve, pollsters must address the methods that they use to poll and ensure that no voter group is favoured in the polls. At present, there appears to be huge favourability towards Democrat voters when completing polls, however pollsters must also be wary that they do not shift methods too much so as to overstate the influence of Republican voters either.

The Bradley Effect may be another reason why the polls were so inaccurate in recent elections concerning Donald Trump. Originally named after Tom Bradley, a Californian politician who was polling much better than his opponent yet lost his 1982 gubernational election, the Bradley Effect details the impact of social desirability bias on answering polls (TIME magazine, 1982). This bias occurs as survey respondents answer questions in a manner that they think will be viewed favourably by others, rather than answering questions truthfully and honestly. In the case of Bradley, groups of white people did not support his policies and did not vote for him, but felt pressure to tell pollsters that they would vote for him to not be seen as racist. In the case of Trump, his unpopularity (FiveThirtyEight, 2024) may lead to a similar effect as people feel it is not socially appropriate to announce they are voting for him. The Bradley Effect is particularly important in the case of male voters in the 2016 and 2024 election, when Trump was competing against a woman on both occasions. This may also be the case for socially conservative immigrant voters; who society expects would dislike Trump due to his harsh immigration policies, but who support some of Trump’s other social policies so eventually vote for him at the polls (SBS News, 2024). 

Despite relative failure by polling firms to predict the election, the betting market saw much greater accuracy. By Tuesday evening, after voting was completed but counting had not yet started, Trump was given a 90% chance of victory by Kalshi and PredictIt, and a 93% chance of victory by Polymarket (The Guardian, 2024). Even a week before the election, PredictIt gave Trump a 60% chance of victory (PredictIt, 2024), meanwhile Polymarket had Trump polling better than Harris for the entire month before the election (Polymarket, 2024). Even traditional betting sites such as Sky Bet had Trump as favourite to win. On election day, Trump was given odds of 4/7 by Sky Bet (2024), implying a 63.6% chance of victory.

One of the main explanations for betting markets being better than pollsters at predicting the election is that the betting markets are less susceptible to the Bradley Effect. By asking what other people would do, betting markets mitigate the social desirability bias that has such a drastic effect on polls. Betting markets also provide a balanced perspective on expectations. A Democrat voter may still place a bet on Trump winning the election if they want to make money. It would be irrational for a person to place a bet on the person they think will lose. 

Despite their strengths, betting markets have some limitations. The main limitation of betting markets is that they tend to consider who will win the popular vote rather than who will win the election. This is because people tend to consider the way that the majority of other people will vote; however do not think deeply about the impacts particular swing state trends may have on the Electoral College. As such, the 2016 election betting market was wildly inaccurate. PredictIt gave Clinton an 82% chance of victory the day before the election (PredictIt, 2016). Even traditional betting sites were thrown, with Paddypower giving Clinton 1/6 odds on the eve of the election, implying an 85.7% chance of victory (Moore J., 2016).

To conclude, it is clear that traditional pollsters currently have a bias towards the Democrats. There are two main reasons behind this bias. Firstly, a greater usage of online polls means that Democratic voters are likely to complete more polls than Republican voters, which overstates the probability of a Democrat victory. The other factor behind the bias is the Bradley effect, where people are influenced by social desirability bias to say they are not voting for Trump, even if they do in fact vote for Trump. Betting markets tend to be better at avoiding these polling biases; however their lack of nuance and state-by-state input means that these markets can fail if the presidential winner does not win the popular vote. The primary reform needed for polling systems is a greater use of mixed-methods polling. In practice, this means using telephone, online, and in-person polling to capture a better representation of the electoral roll. I would also recommend betting companies encourage more state-by-state analysis and betting, to produce a more accurate picture of what the final Electoral College may look like.

References

BBC News. (2024). Why Gen Z & Millennials are hung up on answering the phone. https://www.bbc.co.uk/news/articles/crgklk3p70yo

BBC News. (2020). US election 2020 polls: Who is ahead – Trump or Biden? https://www.bbc.co.uk/news/election-us-2020-53657174

BBC News. (2016). US election poll tracker: Who is ahead – Clinton or Trump? https://www.bbc.co.uk/news/election-us-2016-37450661

Financial Times. (2024). Polls open after Kamala Harris and Donald Trump finish their final push. https://www.ft.com/content/64b240f4-205d-4e76-801c-a820e858ca16

FiveThirtyEight. (2024). Who’s ahead in the national polls? https://projects.fivethirtyeight.com/polls/president-general/2024/national/

FiveThirtyEight. (2024). Do Americans have a favourable or unfavourable opinion of Donald Trump? https://projects.fivethirtyeight.com/polls/favorability/donald-trump/

FiveThirtyEight. (2020). Who’s ahead in the national polls? https://projects.fivethirtyeight.com/polls/president-general/2020/national/

FiveThirtyEight. (2017). The Real Story Of 2016. https://fivethirtyeight.com/features/the-real-story-of-2016/

FiveThirtyEight. (2016). Who will win the presidency? https://projects.fivethirtyeight.com/2016-election-forecast/

Moore, J. [@JimMooreJourno]. (2016, November 8). Here at Ladrokes HQ. Market going heavily for Clinton. 1-6 near shortest she has been… https://x.com/JimMooreJourno/status/796118291542581249

New York Times. (2024). Election 2024 Polls: Harris vs. Trump https://www.nytimes.com/interactive/2024/us/elections/polls-president.html

Pew Research Centre. (2019). What our transition to online polling means for decades of phone survey trends. https://www.pewresearch.org/short-reads/2019/02/27/what-our-transition-to-online-polling-means-for-decades-of-phone-survey-trends/

Polymarket. (2024). Presidential Election Winner 2024. https://polymarket.com/event/presidential-election-winner-2024?tid=1731682729434

PredictIt. (2024). Who will win the 2024 US presidential election? https://www.predictit.org/markets/detail/7456/Who-will-win-the-2024-US-presidential-election

PredictIt. (2016). Who will win the 2016 U.S. presidential election? https://www.predictit.org/markets/detail/1234/Who-will-win-the-2016-US-presidential-election

SBS News. (2024). ‘Ugly’ comments won’t stop diverse voters from supporting Trump. This is why. https://www.sbs.com.au/news/article/ugly-comments-wont-stop-diverse-voters-from-supporting-trump-this-is-why/kgnp1xale

Silver, N. [@NateSilver538]. (2024, November 5). It’s published. We ran 80,000 simulations tonight. Harris won in 40,012. https://x.com/NateSilver538/status/1853673781350260902

Silver, N. [@NateSilver538]. (2024, October 20). A sweep of the swing states is more likely than people might assume… https://x.com/NateSilver538/status/1848060370268938538

Sky Bet. [@SkyBet]. (2024, November 5). The odds of Donald Trump winning the 2024 US Election… https://x.com/SkyBet/status/1853799440718479758

Statista. (2024). Leading candidates for the 2024 presidential election in the United States as of October 2024, by age. https://www.statista.com/statistics/1422251/top-2024-presidential-candidates-age-us/

Stewart, R. [@RoryStewartUK]. (2024, November 4). Kamala Harris will win comfortably because… https://x.com/RoryStewartUK/status/1853325635747266782

The Economist. (2024). Donald Trump vs Kamala Harris: who’s leading the polls? https://www.economist.com/interactive/us-2024-election/trump-harris-polls

The Guardian. (2024). Betting markets swing hard to Trump with early election rallies. https://www.theguardian.com/us-news/2024/nov/05/trump-betting-markets-election-results

The Harvard Gazette. (2017). The puzzle in politics and polling. https://news.harvard.edu/gazette/story/2017/03/nate-silver-says-conventional-wisdom-not-data-killed-2016-election-forecasts/

The Times. (2024). Why were the polls wrong about Donald Trump once again? https://www.thetimes.com/world/us-world/article/why-were-polls-wrong-about-donald-trump-once-again-9kls6fqv7

TIME Magazine. (1982). Fighting the Last War. https://web.archive.org/web/20080603043839/http://www.time.com/time/magazine/article/0,9171,949651-3,00.html