Science does not always make sense

The story of the week for many people was the new alcohol guidelines issued by the UK’s chief medical officers.  In 1995, the recommended weekly upper limit for men was set at 21 units, or around eight pints.  This has now been slashed to only 14 units.

We might imagine that this drastic reduction is based upon some important advance in medical knowledge.  But our cultural cousins in Ireland, Australia and New Zealand do not seem to have noticed it.  They still have recommended limits very close to our 1995 one.  Large swathes of America retain a puritanical mistrust of alcohol, and even pensioners are required to certify their age in many states before they can order a drink.  But the “health limit” for men in the United States is 25 units.  In Spain it is 35.

The methodology of science may seem an abstract subject, but it has important practical implications.  The point in question is: can your conclusions be replicated by other scientists? Medical officers in the UK have used scientific evidence to pronounce about “safe” alcohol limits.  Their Spanish counterparts have done the same, and have come up with a number two and a half times as large.  The conclusions are completely different in the two cases.

Replicability is currently a hot topic in science.  For example, there is a serious crisis in psychology.  It seems that most of the conclusions they draw from their experiments cannot be reproduced when the tests are repeated.  We have all read media features under the headline “six steps to happiness”, or some such compelling title, based on academic psychology.  But we need to take them even less seriously in future.

Science and Nature are the top two scientific journals in the world.  Last August, Science had an article which attempted to reproduce the results of 100 experiments published in leading psychology journals.  The original teams collaborated with the replicators, a fact which should enhance the rate of replicability.  In fact, only 36 per cent of the attempted replications led to results which were statistically significant.  Further, the average size of the effects found in the new studies was only half that reported in the original studies.  The lead author, Brian Nosek, commenting on the paper in Nature, said that there is no way of knowing whether any individual paper is true or false from this work!

Economics has made progress in facing up to this crucial issue.  Can your result be repeated by someone else?  Many leading journals now insist that the data sets and even the code used to generate findings are posted on line.  But there is still a long way to go to get economists to take replication as seriously as do physicists.  A few years ago, my own company created a competition, with a decent prize, for the best paper on replicating an article already published.  In the first year we got five entrants, the second just two, and in the third only one.  Economics must avoid the current pitfalls in medicine and psychology.

Paul Ormerod 

As Published in CITY AM on Wednesday 13th January 2015

Image: Pour by jenny downing licensed under CC BY 2.0

Share this post



t: +44 020 8878 6333

Alex O’Byrne, Associate at Volterra, is an experienced economic consultant specialising in economic, health and social impact, economic strategy, project appraisal and socio-economic planning matters.

Alex has led the socio-economic and health assessments of some of the most high profile developments across the UK, including Battersea Power Station, Olympia London, London Resort, MSG Sphere and Westfield. He has significant experience inputting to EIAs and s106 discussions as well as drafting economic statements, employment and skills strategies and affordable workspace strategies.

Alex is also experienced at economic appraisal for infrastructure. He was project manager of the economic appraisal for the City Centre to Mangere Light Rail in Auckland. He also led the economic and financial appraisals of the third tranche of the Transport Access Program for Transport for New South Wales, in which Alex developed and employed innovative methodological approaches to better capture benefits for individuals with reduced mobility.

He is interested in the limitations of current appraisal methodologies and ways of improving economic and health analysis to ensure it is accessible to as many people as possible. To this end, Alex recognises the importance of transparent and simple to understand analysis and ensuring all work is supported by a robust narrative.

Alex holds a BSc (Hons) in Economics from the University of Manchester and he was a member of the first cohort of the Mayor’s Infrastructure Young Professionals Panel.


Senior Partner

t: +44 020 8878 6333

Ellie is a partner at Volterra, specialising in the economic impact of developments and proposals, and manages many of the company’s projects on economic impact, regeneration, transport and development.

With thirteen years experience at Volterra delivering high quality projects to clients across the public and private sector, Ellie has expertise in developing methods of estimating economic impact where complex issues exist with regards to deadweight, displacement and additionality.

Ellie has significant experience in estimating the economic impact across all types of property development including residential, leisure, office and mixed use schemes.

Project management of recent high profile schemes include the luxury hotel London Peninsula, Battersea Power Station and the Nova scheme at London Victoria. Ellie has also led studies across the country estimating the economic and regeneration impact of proposed transport investments, including studies on HS2 and Crossrail.

Ellie holds a degree in Mathematics and Economics from the University of Cambridge.