Saturday, June 10, 2017

10 Key Insights From The UK Election Backed By Data

This weekend's newspapers will be full of hyperbole about the UK election results and what happened. In my view it is important to cut through opinion and look at the data to provide a clear context.

Here are ten key data backed insights about the UK election:

1. Despite a poor campaign the Tories increased their votes and vote share

The campaign run by Theresa May has been roundly criticised by all sides. It is claimed the campaign was the reason why May only won 318 seats, 12 less than David Cameron won at the previous election. However, May significantly increased the Conservative votes and vote share as we can see below.


May's vote share is exactly the same as that Margaret Thatcher achieved in her 1983 landslide victory.

In 1983 the vote translated into 397 seats because the opposition was split. Labour received just 27.6% and the SDP received 25.4%.  This time Labour attracted votes from all other parties and achieved 40% of the votes which allowed them to take seats from the Conservatives and reduce their majority.

Put into this context despite all its shortcomings the campaign did increase both absolute votes and vote share, It is difficult to know if even a great campaign would have increased the vote share above what Thatcher won at the height of her popularity.

2. Labour attracted votes from all parties and particularly young people

While May did well to increase her votes and vote share, Jeremy Corbyn increased Labour's votes and vote share by a far greater extent.


Corbyn performed significantly better than Gordon Brown or Ed Miliband. In fact Corbyn almost matched the 40.7% Tony Blair achieved in 2001 and much better than Blair's 35.2% in 2005.

Corbyn was able to reach across parties to attract votes from UKIP, Lib Dems, Greens and the SNP. What he also did was mobilise and gain support from young people in a way that has not happened before. Which leads us nicely on to ....

3. Age is the big divide in UK politics

I previously reported on a YouGov survey that found that for every 10 years older a voter is, their chance of voting Tory increases by around 8%. See the chart below.


The Financial Times published a chart yesterday which showed how polarised the age divide has become in recent years.


This is not a surprise as the young generation is feeling increased like the left behind generation. While we don't have the youth unemployment of European countries, younger people feel they have been disadvantaged. Their parents could expect free higher education, secure jobs with good pension schemes, access to housing both social and private, and to retire in their 60s, some in their 50s. These days younger people have to pay high tuition fees, have less secure employment, poorer pension schemes, difficulty in accessing housing due to high prices, rents and deposit requirements, and having to work into their seventies.

What is more of a surprise is that there has not been a reaction from younger people before. In this election young people appear to have become more engaged and to have voted in much larger numbers.

4. Young people have become more engaged and voted in higher numbers

I previously wrote about how young people could increase the Labour vote significantly. Traditionally younger people have not turned out in the same numbers as older generations. As we can see below only 43% voted in 2015.


The Evening Standard ran a survey before the election which reported that 63% of 18-24 year olds said they would vote this time and that 68% of them would vote Labour.

There have been reports of much higher turnout of young people this time. Figures such as 72% have mentioned by David Lammy MP but there is no actual data to back this figure at this time. There is a lot of anecdotal evidence from constituencies of higher turnout and an expectation we will see higher figures. The overall turnout was 68.7%, the highest since 1997.

However, while we wait for the numbers there is little doubt that younger people became far more engaged in this campaign and it is likely that turnout of 18-24 year olds was well north of 43%.

5. Social media matters and reaches parts of the electorate that other media cannot reach

This election has seen a lot of discussion about the role and impact of social media. There are questions over how much impact social media posts and sharing have on voting intention. However, what is not in doubt is that people engage with political posts and content on social media. Below are some of the most interacted with posts on social media, data from BuzzSumo.



The Jamie Oliver rant about the Tories school meals policy got over 100,000 shares and 10m views. The Stormzy video mash up got over 8.5m views. By contrast the BBC One Show with Theresa May had approximately 5m viewers. Remember also the highest ever number of viewers for the BBC's Question Time was 8m viewers back in 2009.

Facebook pages such as Another Angry Voice gain tens of thousands of shares and millions of views. BuzzFeed named the author the most influential UK political journalist and I previously covered my views on why this site's content goes viral with its clever memes and tribal reporting. In a similar way The Canary is highly tribal and gets huge engagement with young Labour supporters.

Jeremy Corbyn is also very popular on Facebook, 1.2m people have liked his page and his post yesterday got over 254,000 likes, shares and comments. Already over 5.5m people have viewed his video. His page is the most popular UK political page on Facebook. If this was a Facebook election Corbyn would have won comfortably.

A 2016 YouGov survey found that 28% of 18-to-24-year-olds got their main news from social media, compared with 24% for TV. My instinct is that this is increasing.

Social media has become a way that young people engage with politics and Corbyn and Momentum have become very effective at using the network with clear messages, live video, and memes. Another effective tactic has been rebutting scare stories in the Mail and the Sun with Facebook posts. This post below was used to effectively to mock stories of Corbyn's previous meetings with terror groups. It was shared over 60,000 times on Facebook.


6. There is a cultural divide reflected, but not defined, by Brexit

While not quite as marked as the age divide there is a cultural divide in the UK which is not the traditional class divide, which you might characterise as Labour being the party of the working class and the Tories being the party of the middle class.

At this election the Conservative vote increased by no fewer than nine points in seats with the highest proportion of working class voters. This is linked to the Brexit issue where Leave voters were overwhelming working class.

There is a cultural divide rather than a class divide. David Goodhart has controversially attempted to define two groups he names the 'anywheres' (roughly 50 per cent of the population) and the 'somewheres' (around 20-25 per cent of the population).  The New Statesman feels this distinction broadly works.

'Anywheres' dominate our culture and society, are university educated, have professional careers, live in metropolitan areas including London and overseas. “Such people have portable ‘achieved’ identities,” Goodhart says, “based on educational and career success which makes them . . . comfortable and confident with new places and people.”

'Somewheres' are more rooted in their geographical identity, older, less well educated and find the rapid changes unsettling. “They have lost economically with the decline of well-paid jobs for people without qualifications and culturally, too, with the disappearance of a distinct working-class culture and the marginalisation of their views in the public conversation.”

Whatever your views on this theory there is a clear education divide. The survey from YouGov below shows that more educated voters are more likely to vote Labour or Lib Dem.


The FT has done some initial analysis of the actual election results and found a correlation between seats that had a swing to Labour and the number of adults with a degree. See chart below.


You could argue older people are less likely to have a degree and this just reflects the age split but when you adjust for just people over 50 they found a similar pattern.


Education had a particularly marked effect in seats that swung Labour or Conservative.

As the FT says "In short: better-educated people tend to vote for leftwing or centrist causes, while those who never went to university are more likely to vote for rightwing or populist parties."

7. People don't want more austerity and inequality

Polling by Ipsos Mori at the end of May 2017 asked people to pick out multiple issues that are facing Britain.
  • 61% highlighted the NHS 
  • 45% Brexit
  • 27% education/schools
  • 25% immigration
When asked to pick a single issue the NHS ranked second after Brexit. The latest British Social Attitudes survey found 93% of people recognise that the NHS is facing a funding crisis – and increasing taxes was seen as the single most popular solution.

There appear to be clear concerns about the ongoing impact of austerity cuts on public services. Labour's manifesto tapped into these concerns but implied these could be reversed, along with building hundreds of thousands of new homes and scrapping tuition fees, by just increasing tax for the top 5% and borrowing. I personally still have a concern that people in the UK want Scandinavian public services without Scandinavian taxes which is unrealistic. There also a limit to how much you can raise taxes without actually reducing tax revenues.

When it comes to income inequality the British Social Attitudes survey found that over 80% thought the income gap had become too large. The Labour Party's core message "For the many. Not the few." resonated with people and was used well in their campaign messages.


This brings us nicely on to....

8. Campaigns and manifestos still matter 

As we noted above, May's campaign was roundly condemned although Conservative support finished fairly much where it started at the beginning of the election, as we can see from the Daily Telegraph poll tracker.


However, the Labour campaign made a significant difference, increasing support from below 30% to 40% over the course of the campaign.

The Labour manifesto has been widely praised as tapping into key concerns while the Tory manifesto was seen as a disaster. You might be surprised therefore when you look at the responses to YouGov's survey results when they asked people which manifesto policies they supported. Four of the five most most supported policies were actually from the Conservative manifesto.


The real issue for the Conservatives was that many of their manifesto proposals were deeply unpopular and these were the policies that got media attention, both mainstream and social media. For example, their policies on paying for social care and taking away school meals (we have seen Jamie Oliver's response got over 10m views).

Thus it appears that unpopular policies have a far greater impact than popular policies but it also raises a real issue about campaign communication. The Labour Party cut through with the 'For the many, not the few' message and with a clear vision about investing in public services by raising taxes for the rich.

9. Return of two party politics in England and no overall majority

In England we have seen the return of two party politics with a vengeance. 87.5% of votes went to the two main parties.


This explains why despite May doing better than Cameron, and even as well as Thatcher, and why despite Corbyn doing as well and better than Blair, neither won a majority. There is no longer a split opposition and if this persists it will make it very hard for either party to win a majority in future. 

In the UK 69 seats were won by minor parties, thus to get a majority the winning party has to get at least 69 seats more than the main opposition party and/or take seats away from minor parties, most notably the SNP.

It should also be noted that while things are different in Scotland, Northern Ireland has also become a two party state. Only two parties, Sinn Fein and the DUP, won seats. This time neither the SDLP or the Ulster Unionists won a single seat. 

The Lib Dem vote was 7.6%, the worst since 1970.

10. Opinion polls need to change and YouGov point the way

I have covered extensively YouGov's attempt to build a new election model for predicting opinion and election results. A new model is required as the opinion poll companies got the result badly wrong again apart from Survation, who had a 1% Conservative lead. The average Conservative lead across the companies was 7.5%, some had the Conservatives with a 10 point lead.

Opinion polls shape the narrative around an election and must therefore be accurate and used with caution. Using widely inaccurate polls is akin to fake news in shaping public opinion.

YouGov has adopted data science techniques called Multilevel Regression and Post-stratification (or 'MRP' for short) for its new model. In simple terms they use the poll data from 50,000 people for the preceding seven days to estimate how a voter with specified characteristics will vote e.g. Conservative, Labour, or another party. They look at a wide range of respondent details including previous voting intentions, age etc.

Then using data from the UK Office of National Statistics, the British Election Study, and past election results, YouGov estimates the number of each type of voter in each constituency.

This model produced the most accurate seat estimates as we can see below.


Unfortunately YouGov lost confidence in this new model and the day before the election said the Conservatives had a 7% lead and would win an increased majority based on their traditional polling methods. This appears to be a classic case of opinion poll companies herding together, so at least they can say they were all wrong and not stand out as the one company that got it badly wrong.

Hopefully they will now have greater confidence and I suspect we will see more investment in these new data science approaches looking at the voter makeup and characteristics in each constituency.

One thing we have seen is that you cannot project national results based on uniform national swings or even regional based swings. A good example is Brighton Pavilion where I live and which I know reasonably well. A number of models had Labour winning the seat based on the national swing to Labour and the high numbers of students and young people in the constituency. However, there are many specific factors in the constituency, not least that Caroline Lucas is the high profile leader of the Green Party. In fact she smashed her previous result taking over 50% of all votes. There was actually a swing against Labour and the Conservatives.


I think there is something here about tribes. There is definitely a Brighton tribe and Caroline is the leader of the tribe. I loved Grayson Perry's analysis of UK tribes and culture, he was definitely on to something. Maybe he should become a political forecaster.

Finally...

For those who have stuck with it I am sorry this was such a long post. There will be more interesting insights we will gain from the election data as it becomes available but for now I hope this has been useful, possibly even insightful or at least given you pause for thought.