Science does not always make sense

The story of the week for many people was the new alcohol guidelines issued by the UK’s chief medical officers.  In 1995, the recommended weekly upper limit for men was set at 21 units, or around eight pints.  This has now been slashed to only 14 units.

We might imagine that this drastic reduction is based upon some important advance in medical knowledge.  But our cultural cousins in Ireland, Australia and New Zealand do not seem to have noticed it.  They still have recommended limits very close to our 1995 one.  Large swathes of America retain a puritanical mistrust of alcohol, and even pensioners are required to certify their age in many states before they can order a drink.  But the “health limit” for men in the United States is 25 units.  In Spain it is 35.

The methodology of science may seem an abstract subject, but it has important practical implications.  The point in question is: can your conclusions be replicated by other scientists? Medical officers in the UK have used scientific evidence to pronounce about “safe” alcohol limits.  Their Spanish counterparts have done the same, and have come up with a number two and a half times as large.  The conclusions are completely different in the two cases.

Replicability is currently a hot topic in science.  For example, there is a serious crisis in psychology.  It seems that most of the conclusions they draw from their experiments cannot be reproduced when the tests are repeated.  We have all read media features under the headline “six steps to happiness”, or some such compelling title, based on academic psychology.  But we need to take them even less seriously in future.

Science and Nature are the top two scientific journals in the world.  Last August, Science had an article which attempted to reproduce the results of 100 experiments published in leading psychology journals.  The original teams collaborated with the replicators, a fact which should enhance the rate of replicability.  In fact, only 36 per cent of the attempted replications led to results which were statistically significant.  Further, the average size of the effects found in the new studies was only half that reported in the original studies.  The lead author, Brian Nosek, commenting on the paper in Nature, said that there is no way of knowing whether any individual paper is true or false from this work!

Economics has made progress in facing up to this crucial issue.  Can your result be repeated by someone else?  Many leading journals now insist that the data sets and even the code used to generate findings are posted on line.  But there is still a long way to go to get economists to take replication as seriously as do physicists.  A few years ago, my own company created a competition, with a decent prize, for the best paper on replicating an article already published.  In the first year we got five entrants, the second just two, and in the third only one.  Economics must avoid the current pitfalls in medicine and psychology.

Paul Ormerod 

As Published in CITY AM on Wednesday 13th January 2015

Image: Pour by jenny downing licensed under CC BY 2.0

Share this post

Share on facebook
Share on google
Share on twitter
Share on linkedin
Share on pinterest
Share on print
Share on email



t: +44 020 8878 6333

Ellie is a partner at Volterra, specialising in the economic impact of developments and proposals, and manages many of the company’s projects on economic impact, regeneration, transport and development.

With thirteen years experience at Volterra delivering high quality projects to clients across the public and private sector, Ellie has expertise in developing methods of estimating economic impact where complex issues exist with regards to deadweight, displacement and additionality.

Ellie has significant experience in estimating the economic impact across all types of property development including residential, leisure, office and mixed use schemes.

Project management of recent high profile schemes include the luxury hotel London Peninsula, Battersea Power Station and the Nova scheme at London Victoria. Ellie has also led studies across the country estimating the economic and regeneration impact of proposed transport investments, including studies on HS2 and Crossrail.

Ellie holds a degree in Mathematics and Economics from the University of Cambridge.