Samizdata

A little over a year ago I asked the following question:

Has the day come when election polls are nearly always right?

Famously, in the last US presidential election, Nate Silver correctly predicted the winner of all 50 states and the District of Columbia. His prediction for the election before that was correct for 49 out of 50 states.

Both times, I had hoped it would turn out otherwise. My hopes had been a little higher than they should have been because of the residual glow from the Shy Tory factor, first exhibited to a dramatic extent in the 1992 UK general election and still apparent, though in lesser degree, for several elections after that. I had known about that factor in my guts before that election, from listening to people on the tube, and had correctly guessed the final result would be more Conservative than the polls claimed. As the results came in I did not rejoice that the Government would be Conservative, but I did rejoice that the Chattering Classes had been confounded, their bubble burst, their conversational hegemony broken open and their flary-nostrilled noses put out of joint. Yeah.

Unfortunately not-yeah since then. I haven’t eaten a hearty post-election breakfast with schadenfreude sauce about the polls for many a year now. George Bush winning in 2004 was splendid fun, of course, but it was no great surprise to anyone who had been paying attention. The polls had given him a consistent small lead for months before the election.

Betteridge’s law of headlines strikes again. That day had not come. The polls in the General Election of 2015 were wrong, wrong, wrongety-wrong, wrongbert, wrongble and wrong.*

As was I, but least I had the nous to put in a question mark.

So, elections just got interesting again. Goody! But none of the articles I have yet seen adequately explain why the Shy Tory effect was successfully allowed for by the pollsters in the UK General Elections between 1992 and 2015, only to burst forth again now, nor why political polling in the US has generally managed to factor in Shy Republicans just fine. Except for the 2014 midterms.

The one place where the UK polling companies did fairly well this time round was Scotland, although they still underestimated the scale of the SNP’s triumph. Wishful thinking led me to suppose that the estimates being chucked around of 48 seats for the SNP were exaggerated; in the event they were too cautious. Going back to 2011, it is part of Scottish Nationalist mythology that the victory of the SNP in the Holyrood election of 2011 was completely unpredicted by the polls. However the very last polls were quite close to the actual result when it came to the constituency vote, but much less close when it came to the regional vote in the Scottish Parliament’s semi-proportional voting system. Probably the polls recorded a shift of opinion in the last few weeks of the campaign, which is all you can ask of them. When you think about it, polls cannot predict anything; the people who look at them do that. The final polls for the Scottish referendum were out by a not-bad 5% or so, in the usual direction of underestimating the small-c conservative side.

All in all, a British or American polling company attempting to sell its wares to interested political parties or news organizations on May 6th 2015 could have made a fair case that they were on top of the Shy Tory problem. So what happened on May 7th? What will happen on November 8th 2016, and will we have any idea beforehand?

*This is funny but nothing to do with this post. Americans and people under 50: don’t ask.