By Matthew Goodwin
‘Trying to predict the future’, said the famous management consultant Peter Drucker, ‘is like trying to drive down a country road at night with no lights while looking out the back window’. Look back on 2015 and it seems that nearly all of those who tried to forecast this year's political events were passengers on this crazy journey, hurtling down a dark country lane while confidently heading towards a ditch.
Just think about how much the pundits and analysts got wrong. Let’s start with the biggest moment in the political calendar, Britain’s general election. Columnists, led by pollsters and never questioning their data, collectively bought into the idea of a hung parliament. When the election website May2015 listed predictions from newspapers, bookies and number crunchers, every single one predicted a hung parliament. As Britain entered 2016, David Cameron was supposed to be finished and Prime Minister Miliband was supposed to be head of a minority Labour government or some kind of left-wing coalition. Ok, there were one or two exceptions to the wisdom of the crowd. But even those columnists who dared to mutter out loud about the possibility of a Conservative majority were going on nothing more than a hunch. There was nothing scientific about it. And this is what made it all so embarrassing. Today we have more data than ever before on British politics and yet people who are supposed to know more than anyone else, collectively failed to see what was coming.
And I include myself in that, alongside the other academics and election nerds who should really know their stuff. Shortly before the election, Britain’s top academics who forecast elections, met in London to share their fancy statistical models. But, once again, they were all wrong. Not one predicted a majority government. Of the eleven forecasts nearly half had Labour winning the largest number of seats. On average they predicted a dead heat with Labour on 283 seats and the Tories on 278. But in the end Labour nosedived to reach 232 seats while the Tories surprised us all by taking 330 (not a single forecaster thought Cameron would pass the 300 mark).
It wasn’t only the election that we got wrong, of course. Nearly all of those who make their living writing about politics also failed to predict the rise of new insurgents and the collapse of older parties. Many people only grasped the formidable strength of the Scottish National Party’s revolt late on while almost everybody failed to grasp the durability of Ukip. Ever since Nigel Farage and his self-anointed ‘People’s Army’ stormed into the limelight by winning the European elections in 2014, newspaper columns were filled with predictions that the flash party was just that – a flash-in-the-pan. It took observers far longer than it should have to recognise that the bulk of Ukip’s support was coming not from middle-class Tories in the Shires but the disillusioned working-classes who might have once voted Labour. Both the SNP and Ukip had a far greater impact than many assumed. While Ukip ultimately failed to win more than one seat, almost nobody foresaw that the party would actually hold on to nearly four million voters. In a similar vein, hardly anyone predicted that the SNP would get to 56 seats or meanwhile that the Liberal Democrats would be left with only eight (not a single one of those predictions on the May2015 website thought Clegg’s party would have less than 20 MPs). Like the morning after the night before it was all rather awkward.
But then things got worse, a lot worse. If the election had revealed a fundamental misunderstanding of the electorate then the race for the Labour leadership uncovered a total disconnect between pundits and the main opposition party. When Jeremy Corbyn threw his hat into the ring he was presented as a joke candidate, a reluctant challenger or Trotskyite no-hoper. But once again the commentariat had failed to accurately take the pulse of the nation. The consensus was reflected in the answer of one newspaper to the question: 'What are his chances?' 'Bleak', came the reply. The bookies put him on distant odds of 100/1. Then he won.
Then, to bring 2015 to a close, came the first by-election of the new parliament, held in the rock-solid safe Labour seat of Oldham West and Royton. This time around there was much talk by election ‘analysts’ that Labour’s majority would be reduced to less than 3,000 votes, and that Ukip might even win the seat, ending Corbyn's regime before it had even reached 100 days. But, in the end, all of that was wrong too. Labour not only held on to the seat but increased its share of the vote. Wrong, again.
So, all in all, 2015 was not a great year for pundits, forecasters, or analysts. The obvious question is why was everybody so off? One explanation takes us back to Drucker and the point that people spent too long looking backwards, rather than forward. It seems to me that too few have grasped how the fundamentals of British politics are changing – how public loyalty to the main parties is in decline, how issues are now more important than old tribal allegiances and how the rise of new concerns such as nationalism and immigration cut across old party divides. All of this has made our politics more volatile, chaotic and, ultimately, unpredictable. Of course, I could be wrong. But either way, with a fresh set of elections in May and a national referendum around the corner we will all need to raise our game in 2016.
Matthew Goodwin is professor of politics at the University of Kent and tweets @GoodwinMJ. He is co-author of the new book UKIP: Inside the Campaign to Redraw the Map of British Politics (Oxford University Press).
The opinions in Politics.co.uk's Comment and Analysis section are those of the author and are no reflection of the views of the website or its owners.