Why the same flaws afflict economic data as political opinion polls

Who will win the US presidency?

Opinion polls have got a bad name in Britain, at least. During the 2010 general election campaign, many suggested that Gordon Brown could still continue in power in a minority government or coalition. The polling record in the 2015 campaign was even worse. Most polls showed the two main parties neck and neck, and even the exit poll on the day did not predict the overall majority achieved by David Cameron.

Conventional, survey-based polling is encountering fundamental problems, and they’re not confined to the narrow area of trying to discover political opinions. They are much more general. Household surveys are the source of a wide range of statistics which guide policy, such as official rates of poverty, inflation, and, in the United States, unemployment rates and health insurance coverage. They are also a primary source of data for a lot of economic research.

But an article on their accuracy by Bruce Meyer of the University of Chicago and his colleagues in the Fall 2015 issue of the prestigious Journal of Economic Perspectives concludes starkly that “the quality of data from household surveys is in decline”. The title of the paper says it all: “Household surveys in crisis”. Households have become less likely to answer surveys at all, those that do respond are less likely to answer all the questions, and their answers are becoming less accurate.

In the US government’s consumer expenditure survey, for example, non-response rates have risen from 15 per cent in the mid-1980s to 35 per cent now. In surveys designed to measure poverty, 25 years ago between 10 and 20 per cent of respondents failed to answer all the questions, but this “item non-response” rate is now as high as 40 per cent.

In a lengthy discussion, the authors point out that it is difficult to pinpoint exactly why these things have happened, precisely because any survey designed to answer the question would itself be unreliable! But they suspect that an important reason is because “talking with interviewers, once a rare chance to tell someone about your life, is now crowded out”. People have so many opportunities to give feedback via email and social media that they just cannot be bothered with conventional surveys.

These trends create formidable problems for polling organisations in trying to correct for any biases which might exist in any particular survey or sample. Adjustments to a sample which might have worked 20 years ago to give an accurate picture of the population as a whole no longer have the same validity. This is exactly the problem that the political pollsters in the UK have encountered.

Against this background, what are we to make of the opinion polls in the run up to the American presidential election?

On the face of it, Hillary Clinton does seem to have a small lead over Donald Trump, but her margin is narrowing. Perhaps the most scientific prediction would be to just toss a coin.

Paul Ormerod 

As published in CITY AM on wednesday 14th September 2016

Image: Hillary Clinton by Roger H. Goun / Donald Trump by  Gage Skidmore is licensed under CC BY 2.0

Share this post



e: aobyrne@volterra.co.uk
t: +44 020 8878 6333

Alex O’Byrne, Associate at Volterra, is an experienced economic consultant specialising in economic, health and social impact, economic strategy, project appraisal and socio-economic planning matters.

Alex has led the socio-economic and health assessments of some of the most high profile developments across the UK, including Battersea Power Station, Olympia London, London Resort, MSG Sphere and Westfield. He has significant experience inputting to EIAs and s106 discussions as well as drafting economic statements, employment and skills strategies and affordable workspace strategies.

Alex is also experienced at economic appraisal for infrastructure. He was project manager of the economic appraisal for the City Centre to Mangere Light Rail in Auckland. He also led the economic and financial appraisals of the third tranche of the Transport Access Program for Transport for New South Wales, in which Alex developed and employed innovative methodological approaches to better capture benefits for individuals with reduced mobility.

He is interested in the limitations of current appraisal methodologies and ways of improving economic and health analysis to ensure it is accessible to as many people as possible. To this end, Alex recognises the importance of transparent and simple to understand analysis and ensuring all work is supported by a robust narrative.

Alex holds a BSc (Hons) in Economics from the University of Manchester and he was a member of the first cohort of the Mayor’s Infrastructure Young Professionals Panel.


Senior Partner

e: eevans@volterra.co.uk
t: +44 020 8878 6333

Ellie is a partner at Volterra, specialising in the economic impact of developments and proposals, and manages many of the company’s projects on economic impact, regeneration, transport and development.

With thirteen years experience at Volterra delivering high quality projects to clients across the public and private sector, Ellie has expertise in developing methods of estimating economic impact where complex issues exist with regards to deadweight, displacement and additionality.

Ellie has significant experience in estimating the economic impact across all types of property development including residential, leisure, office and mixed use schemes.

Project management of recent high profile schemes include the luxury hotel London Peninsula, Battersea Power Station and the Nova scheme at London Victoria. Ellie has also led studies across the country estimating the economic and regeneration impact of proposed transport investments, including studies on HS2 and Crossrail.

Ellie holds a degree in Mathematics and Economics from the University of Cambridge.