As the 2024 UK General Election campaign ends, the broad outcome seems certain. Barring any massive, last-minute change – or an unprecedented, industry-wide poll error – the Labour Party will control a substantial majority of seats in the House of Commons by Friday afternoon.
There seems to be more uncertainty around the outcome of the election in Scotland. Polls using multilevel regression and post-stratification (MRP) approaches to predict outcomes in individual constituencies have suggested anything from eight seats for the Scottish National Party (SNP) to 37, reflecting the SNP winning anything from 14% of Scottish seats to 65%. The two results would come with substantially different consequences for the development of Scottish political narratives in the aftermath of the election and different implications for our analyses of the election in Scotland.
Is this perception of the polls being “all over the place” in Scotland accurate? As ever, it is crucial to look across the range of polls available, rather than focus on the most interesting or unusual. Doing the latter is naturally tempting – not to mention unusual polls make good headlines – but distracts us from the propensity of evidence polls are producing and makes us more likely to make errors ourselves.
In this piece, I cut through the noise and identify the signal in Scottish polling, using the range of data and types of polls available to us to identify the clear story Scottish polling is telling without overstating the certainty of the predictions we can make. Firstly, why pay attention to the polls in Scotland at all?
Accuracy of Scottish Polling
The accuracy of political polls can be assessed in a few different ways. The most straightforward is to measure how closely the estimated voting intention of ‘call polls’ – the final polls in a campaign, conducted right before the election – matches the results of the election.
Across the 2015, 2017, and 2019 general elections, and the 2016 and 2021 Scottish Parliament elections, we had a total of 26 call polls producing a total of 104 vote share estimates across the four main parties. There is a high likelihood that several of these estimates will fall outwith the statistical margin of error. In 2019, for instance, Panelbase (now Norstat) understated SNP support by six points, and in 2021 Savanta understated SNP support on the constituency ballot by 5.7 points.
But broadly speaking, as the table below sets out, the polls in Scotland have been reasonably accurate, in the past two election cycles have been very accurate and are even more accurate when their estimates are used to produce a poll average.
They are not perfect tools. The nature of survey research is such that error is inevitable and outliers happen, which is why single polls should never be taken as gospel and we should not put trust in predictions based on single polls. But as tools for understanding the state of public opinion, they are accurate, particularly when taken in aggregate, and particularly in comparison to anecdote, vox pops, and canvassing returns which (often purposefully) capture unrepresentative samples of the population.
Interpreting Scottish Polling in 2024
There are three main sources of data on how Scots intend to vote in general elections: Scotland-specific polls, Scottish subsamples from national polls, and the vote shares implied by the results of MRP models.
Scotland-specific polls remain the gold standard method for measuring Scottish public opinion, capturing a representative sample of (typically) 1,000 or more Scottish adults, weighted to the demographic and, in many cases, political profile of the Scottish adult population. These are the polls that, in aggregate, accurately called the vote shares of the major parties in the past several election cycles in Scotland.
Scottish subsamples in national polls are substantially less reliable on a poll-by-poll basis than Scotland-specific polling. There are three main reasons for this. Firstly, they consist of much smaller numbers of people, usually 100 or fewer. This means that the statistical margin of error for estimates of voting intention in such subsamples is closer to ±10 percentage points than the roughly three percentage points typically used as a guide to margin of error in a full-size poll.
Secondly, subsamples are not recruited to reflect the population profile within Scotland, and thirdly they are not then weighted to reflect the Scottish population – so we should expect even more variance in the vote shares estimated by subsamples than the statistical margin of error would suggest.
That said, in aggregate, they, too, can provide useful information. The average subsample voting intention for the SNP since June 24th is 30.6%, and the equivalent average in Scotland-specific polling is also 30.6%. For Labour, the two figures are 35.2% and 36%. In fact, the greatest degree of divergence is just 1.9 points, for Reform UK. They offer a useful supplement to much less regular Scotland-specific polls.
MRPs provide a further source of data, albeit the implied vote shares from MRPs are reverse-engineered from constituency-by-constituency turnout and voting intention estimates. Many in-depth explanations of how MRPs work have been published in the past month or so, and we are yet to see how they perform in aggregate as models to predict the outcome of the election, so I will not go into too much depth on how they work or how reliable they might be. It suffices to say that, again, the average vote shares in Scotland that they imply do not diverge significantly from those of Scotland-specific polls – the greatest divergence is 1.3 points for Labour.
We can bring together these three sources of voting intention estimates to model voting intention for each of the parties from 2020 to today, which I do below by fitting weighted[i] LOESS regressions to the vote share estimates of each party and plotting predicted vote shares over time based on those models. The translucent bands on either side of the coloured lines represent prediction intervals, which represent the degree of confidence and potential error inherent in the predicted vote shares for each party.
The overarching story is fairly clear. Labour first broke through in this Parliament around the time of Liz Truss’s premiership, winning over Conservative voters, before eating into the SNP vote after Nicola Sturgeon’s resignation as First Minister. They were neck-and-neck with the SNP by late 2023 and took a clearer – but not decisive – lead following the collapse of the Bute House Agreement and Humza Yousaf’s resignation. A subplot of the campaign period is Reform UK’s gains at the Conservatives’ expense.
Entering the final week of the campaign, these models predict that Labour has a lead of 4.5 points over the SNP:
It’s crucial to bear in mind that the prediction intervals for Labour and the SNP overlap here. In other words, there is the possibility – based on the voting intention estimates we have from Scotland-specific polls, subsamples, and MRPs – that the SNP are ahead of Labour. For example, the polls could be systematically overestimating Labour support. But, by the same gesture, Labour could be even further ahead of the SNP than they appear to be. The predicted values are, based on the data we have, the most likely to reflect the actual levels of support for each party in the voting population, but that likelihood is not 100%.
4.5 points is a relatively narrow lead. Certainly, it is not on the order of magnitude of the Labour lead over the Conservatives across Britain. And so, we should also bear in mind that, in the final days of the campaign, a fairly small swing back to the SNP could cause that lead to evaporate.
Using Scottish Polling to Predict the Election Outcome
Labour narrowly lead the SNP, having won voters over from both the SNP and the Conservatives, and the Conservatives are further losing votes to Reform UK on their right. But what does that mean in terms of the actual election outcome and how many seats each party will win?
Predicting seat results based on polling data is a thorny exercise, and many different approaches exist. Most UK general elections since the Second World War have followed a pattern of uniform national swing (UNS), in which a party losing or gaining a certain share of the vote nationally loses or gains that same share of the vote in each constituency. For example, if the Conservatives lose three points of vote share, they will lose around three points of vote share in each constituency.
But ‘big swing’ elections in which massive numbers of voters shift tend to follow a pattern more closely approximating proportional swing (PS), in which parties lose and gain vote share in constituencies in proportion with their losses and gains nationally – in other words, if the Conservatives lose 20 points nationally, and this is half their vote, they will lose 30 points in a constituency in which they won 60%, and 10 points in a constituency where they won 20%.
More innovative and complex methods have been developed to deal with parties that have small national vote shares, but are heavily concentrated in a few areas, like the Liberal Democrats – strong transition models are a case in point. Regression models are increasingly common, using a much wider range of variables in tandem with polled voting intention to predict constituency outcomes. MRPs are an excellent example of new approaches to predicting constituency outcomes, and we have just had the first stacked regression and poststratification (SRP) in British election history. If nothing else, this has been the most innovative election for the polling industry in an extremely long time.
Again, there are many explainers available for how all these methods work, so I won’t go into any more detail. Poll watchers will have noticed that the more complex methods remain error-prone, and almost every MRP of the campaign has produced evidently odd predictions in at least some constituencies. Much like all political polling, error is part of the game and outliers happen.
As such, we ought to look across the range of models available to us and aggregate them. The simplest way to do this, given how little information we have available about the inner workings of individual, more complex models, is to measure the degree of consensus between models[ii]:
Aggregate Scottish Seat Prediction (2024 General Election)[iii]
Here, I have aggregated 16 models using various modelling approaches. Three are based on the polling average I provided above – one uses UNS, one uses PS, and one is a strong transition model. Four are poll aggregators, eight are MRPs, and one is an SRP. Seat counts reflect the degree of consensus across these models: ‘consensus’ indicates 100% consensus; ‘strong’ reflects at least 90% consensus; ‘leans’ indicates at least 80% consensus, and ‘marginal’ indicates at least 65% consensus.
Toss-ups are seats with less than 65% consensus on what party will win there. Six of these are Labour-SNP contests, three are SNP-Conservative contests, and one is a Labour-SNP-Conservative contest. Considering this, we can suggest the following range of outcomes:
It is, of course, possible that the final result might fall outwith the range suggested above – one outlier model might have the result correct, after all. But again, this is unlikely to happen.
This set of aggregated models gives us a much cleaner range of possible outcomes without overstating the certainty of the predictions. 18 seats are marginal or toss-ups, accounting for nearly a third of the seats in Scotland. These seats are the difference between a narrow Labour win, and a substantial Labour win and SNP collapse.
Cutting through the noise: what do the polls tell us about the election in Scotland?
The mountain of polling that has been produced across the UK, including in Scotland, has often created as much noise as signal during this election campaign. If one is not watching the polls and models closely, the din can be confusing – in quick succession last month, we had an MRP telling us that the SNP would win 15 seats, and another telling us that they would win 37 seats. We have had polls within a week of each other telling us that Labour has a lead of ten points or no lead at all.
Cutting through this by aggregating polling data and modelling helps us to find the signal in the noise, quiet the din and identify a clear story without overstating the certainty we have about the election outcome.
Labour are very heavily favoured to win the most seats in Scotland. The most likely outcome is a Labour lead of around 4.5 points, and a narrow majority. But the close nature of so many of the constituencies means that this is not certain, and the election could be closer than that. Factoring in the potential for poll error, or a shift in the polls in the dying days of the campaign, that level of uncertainty grows.
A little-acknowledged subplot of the campaign is the weakness of the Conservative vote. If the polls are correct, they could win less than 13% of the vote and lose all but one of their seats. Whether this transpires largely depends on whether they can squeeze the Reform UK vote (and, again, whether poll estimates of the Reform UK vote are accurate). If they do, they will be more likely to hold on to seats the SNP hope to gain, lessening the SNP’s ability to compensate for the loss of seats in the Central Belt.
What cannot be argued, unless we see unprecedentedly huge levels of polling error, is that Scottish politics has swung strongly in Labour’s favour over the past two years and electoral momentum is now on their side.
[i] Data sources are weighted according to sample size – for example, the average Scotland-specific poll has ten times the influence of the average subsample, as they have a ten-times larger sample size on average.
[ii] Full list of included models: Uniform National Swing, Proportional Swing, Strong Transition, Poll Aggregators (Electoral Calculus, The Economist, Financial Times, New Statesman), MRPs (Survation, Electoral Calculus, WeThink, Focaldata, YouGov, Savanta, More in Common, Ipsos), SRP (JL Partners).
[iii] Groups of models are weighted to ensure MRPs do not drown out other forms of modelling.