A few years back US academics conducted a study of predictions made by leading political pundits. Of the 26 supposed experts they looked at, just nine had a successful predictive rate higher than 50%. Overall you were just as likely to get an accurate prediction from flipping a coin as you were from listening to the pundits. In some cases a coin toss was a significantly more reliable indicator.
A similar study was conducted in the run up to the 2010 general election. The Independent newspaper asked Britain's leading pollsters for their predictions of the general election result. Of the eight asked, every single one predicted that the Conservatives would get a majority. Every single one was wrong. Of the political commentators asked, just one (Channel Four's Cathy Newman) came close to accurately predicting a hung parliament, and even she hedged her bets.
Reminder: In April 2010, every single pollster and vast majority of commentators predicted an overall Tory majority. pic.twitter.com/ykt8E18tpn— Adam Bienkov (@AdamBienkov) May 4, 2015
A more extensive study was conducted for last year's general election by the Political Studies Association. They asked over 500 leading academics, journalists and pollsters for their predictions of the final result. All three groups predicted an incredibly tight final result, with the Tories only narrowly ahead on vote share. They also predicted the Liberal Democrats would hold on to around 25 seats, despite all the polls suggesting they faced a wipeout. In the end, all three groups were wildly out of line with the final result, with the the Tories gaining an overall majority and the Lib Dems falling to just eight seats. Of all three groups, the academics were the least accurate. Again, flipping a coin would have been a better predictor than listening to the combined expertise of Britain's political specialists.
Today the PSA released another study on the outcome of this month's EU referendums. In this case, the consensus was even more overwhelming. Overall, 87% predicted that Britain will stay in the EU with just 8% saying otherwise. Journalists were the most convinced, with 97% predicting for Remain and just three per cent disagreeing. On average, the experts predicted that Remain would win by 10%.
This, more than any poll released so far, makes me think that we could be heading for Brexit. When the political consensus is this overwhelming, the chances that we are witnessing yet another case of mistaken group-think look worryingly high.
The fact that 97% of journalists think Remain will win, just makes me think we're heading for Brexit. pic.twitter.com/We5Yco8uiD— Adam Bienkov (@AdamBienkov) June 3, 2016
Of course the 'Westminster consensus' isn't always wrong. Matt Singh, who was pretty much the only forecaster to accurately predict the outcome of last year's general election (and was similarly successful in the recent local and regional elections), still believes we are looking at a comfortable Remain win. For the sake of transparency, I should also point out that I was one of the 97% of journalists who told the PSA that I predicted a Remain win. I should also say that if I had to bet all my worldly goods on the result, I would probably still lump for Remain.
However, this study does worry me, especially when combined with polling data which suggests that the fundamentals just might favour the Leave campaign much more than commonly believed. Let's just run through the top two reasons why almost all of Britain's leading political experts could have got this race wrong.
The public are more much concerned with immigration than the economy
An Ipsos-Mori poll released this week found that 38% of the public listed immigration as the top, or one of the top issues facing the country, compared to just 23% who said the same for the economy.
Overall, concern with the economy was at its lowest level since 2008. With the Remain campaign now concentrating almost exclusively on the latter and the Leave campaign now concentrating almost exclusively on the former, the chances of Brexit do appear better than the experts believe.
The Remain coalition looks fragile
Analysis by the Financial Times of YouGov polls reveals that the groups most likely to back either Remain or Leave. As you can see below, Remain voters tend to be much younger and more liberal and left-leaning than Leave voters.
Today's uncontrollable shuddering about the European referendum is brought to you by the FT. pic.twitter.com/kcxgI9kli0— Stephen Bush (@stephenkb) June 2, 2016
Remain voters are also concentrated in London and outside England. With other research suggesting almost half of Labour voters don't even know their party supports staying in the EU, Remain could struggle to get out the voters they need to win. However, the picture isn't all bad for Remainers. The findings also suggest that wealthier and more educated voters are leaning heavily towards Remain, which could potentially cancel out the demographic benefit of older voters leaning towards Leave. Overall though, it looks fairly evenly balanced and certainly doesn't justify the kind of overwhelming consensus of a Remain win that we see in today's study.
So what are we to believe? If we are just going by the polls, then the result looks too close to call. The latest poll of polls suggests the race is now 51% to 49% in favour of Remain. While trust in polls is currently very low, methodological changes made by opinion pollsters in the wake of last year's general election do appear to have worked. Three of the pollsters polling the London mayoral election last month got the final result spot on. If they're similarly accurate this time, then the EU result could be close to a dead heat.
If that turns out to be the case, then putting your trust in the toss of a coin will once again prove to have been a far more reliable indicator of whether Britain leaves the EU, than putting your trust in the combined knowledge of Britain's leading political experts.
Adam Bienkov is the deputy editor of Politics.co.uk
The opinions in politics.co.uk's Comment and Analysis section are those of the author and are no reflection of the views of the website or its owners