Famously, in the last US presidential election, Nate Silver correctly predicted the winner of all 50 states and the District of Columbia. His prediction for the election before that was correct for 49 out of 50 states.
Both times, I had hoped it would turn out otherwise. My hopes had been a little higher than they should have been because of the residual glow from the Shy Tory factor, first exhibited to a dramatic extent in the 1992 UK general election and still apparent, though in lesser degree, for several elections after that. I had known about that factor in my guts before that election, from listening to people on the tube, and had correctly guessed the final result would be more Conservative than the polls claimed. As the results came in I did not rejoice that the Government would be Conservative, but I did rejoice that the Chattering Classes had been confounded, their bubble burst, their conversational hegemony broken open and their flary-nostrilled noses put out of joint. Yeah.
Unfortunately not-yeah since then. I haven’t eaten a hearty post-election breakfast with schadenfreude sauce about the polls for many a year now. George Bush winning in 2004 was splendid fun, of course, but it was no great surprise to anyone who had been paying attention. The polls had given him a consistent small lead for months before the election. In the same year there was an unexpected result in the Spanish general election, but that could be attributed to the the Madrid train bombings three days earlier and the cowardice shown by the Spanish people in their reaction to the attack.
In the years since then I have had the impression that polls have been getting ever more accurate. But my attention has wandered from politics so my impression might be wrong. In recent months the approaching Scottish independence referendum has rekindled the old flame and I have begun to follow the polls. If you want to know, I am of the Unionist persuasion, but it is one of those questions where my libertarianism isn’t telling me which way to steer; and in this post I am not arguing either way. I am just observing that the polls diverge and wobble much more widely than they seem to for either British or US general elections. Is that because it is a referendum rather than an election? I would expect the simplicity of a yes-or-no referendum to make prediction easier, but polling for the voting system referendum in 2011, while correct about the result, did significantly understate the vote to continue with the First Past the Post system, causing my heart to beat faintly once more to the happy rhythm of 1992.
Here is an interesting article by John Curtice, Professor of Politics at Strathclyde University, entitled Scotland’s referendum: can we trust the polls? Mind you, despite being a professor of politics and running a referendum polling blog he does not actually say whether we can or cannot. Quite like old times.