‘Many forecasts are garbage and can be shown to be worse than garbage’
(PRWEB UK) 13 December 2012
‘Majority of forecasters are fools or liars’ says Professor Bent Flyvbjerg in new paper on the inaccurate estimates for major projects
BT Centre for Major Programme Management, Saїd Business School,
University of Oxford
Professor Bent Flyvbjerg, a leading international expert in the management of major projects, delivers outspoken criticism of forecasters in a new paper which demonstrates the considerable inaccuracy of front-end estimates of costs and benefits that are typically used to support decisions on projects. ‘Estimates are commonly poor predictors of the actual value and viability of projects, and cannot be trusted as the basis for informed decision-making. These forecasts frequently misinform decision makers on projects instead of informing them. Some of the inaccuracy comes from genuine forecasting mistakes arising from over-optimism, but some estimates are deliberately misleading, designed to secure financial or political support for a project.’
Professor Flyvbjerg goes further and calls many of these forecasts ‘garbage’ - moreover, expensive garbage, both in terms of the fees of the consultancy firms involved, and in terms of the costs and risks to investors and other stakeholders, that arise from their misrepresentations. ‘Many forecasts are garbage and can be shown to be worse than garbage’ says Flyvbjerg. ‘These reports give the client, investors and others the impression that they are being informed about future demand, or the costs involved in a major project, when they are being misinformed. Instead of reducing risk, reports like this increase risk by systematically misleading decision-makers and investors about the real risks involved.’
Flyvbjerg calls for firm action to be taken by all involved in the sector to address the problem. Clients should ask for their money back when they receive reports which later prove to be significantly inaccurate and misleading. In some cases, he urges them to seek compensation for decisions taken and occasionally, criminal action would be justified, he says. ‘Merely firing the forecaster may be letting them off too easily. Some forecasts are so grossly misrepresented and have such dire consequences that we need to consider suing them for the losses incurred as a result. In a few cases where forecasters foreseeably produce deceptive forecasts, criminal penalties may be warranted’.
He calls upon professional organisations like the PMI, APM, APA, RTPI and others to use their codes of ethics to penalize and possibly exclude members who produce unethical forecasts, and chastises the associations for failing to address this adequately. ‘This needs to be debated openly within the relevant professional organisations. Malpractice in project management should be taken as seriously as malpractice in other professions like medicine and law.’
Above all, such inaccuracy in forecasting is unnecessary, demonstrates Flyvbjerg. ‘Recent research has developed the concepts, tools and guidance on incentives that could help curb both delusional and deceptive forecasts.’ Flyvbjerg’s own work has been central in much of this and in this new paper he sets out a clear case for quality control and due diligence to be applied to the evaluation of front-end forecasts. In a carefully argued and detailed piece, he offers an 8 point due diligence plan which will allow those reviewing project forecasts to evaluate their accuracy as a basis for decision-making and investment.
‘Whether forecasters are unwittingly or deliberately under-estimating the costs, completion times, and risks of projects, and over-estimating their benefits, we need to have a systematic basis for evaluating their findings in order to make informed investment decisions. Given the high cost of major infrastructure projects, the irreversibility of decisions, and the limited availability of resources, this is clearly critical for both public sector and private sector projects. Significantly more accurate forecasts can be produced by looking at the evidence available from previous similar projects which have been already completed – what I call, taking an “outside view”. This seems so simple, but in practice it is transformative and leads to much more accurate forecasting.’
Using the case of an A-train project and drawing on the largest data set of its kind (which includes all large transportation infrastructure projects for which valid and reliable data on forecasting accuracy is available), Flyvbjerg works through his 8 step due diligence plan in detail, evaluating the forecasters findings and demonstrating how it is possible to critically evaluate the accuracy and reliability of forecasts to take decisions from an informed basis. The plan relies upon establishing a statistically reliable benchmark from data available from other similar projects and using this benchmark built on the average performance of the previous projects, to compare against the estimates for a proposed project. It quickly becomes apparent if the forecasts are claiming to meet or exceed the average results of previous similar projects and the reasons can then be scrutinised.
On the initiative of the US Congress, the US General Accounting Office (GAO) has recently adopted Flyvbjerg’s methodology to undertake due diligence on the decision to build a $98.5 billion high speed rail system in California, the most expensive civil construction project in US history. Flyvbjerg is working with the GAO in this task.
Flyvbjerg has also recently been invited to serve as an expert witness in a $150 million class action suit against a forecaster. This is the first such case and may well set a precedent in terms of holding forecasters to account.
To speak with Professor Flyvbjerg about these issues, please contact the press office at Saïd Business School:
Clare Fisher, Head of Press Relations,
Mobile: +44 (0) 7912 771090; Tel: 01865 288968
Josie Powell, PR Coordinator,
Mobile +44 (0)7711 387215; Tel: +44 (0) 1865 288403
Email: josie.powell(at)sbs.ox.ac.uk or pressoffice(at)sbs.ox.ac.uk
Notes to editors
1 About the BT Centre for Major Programme Management
The BT Centre for Major Programme Management is the first research and teaching centre in the world dedicated to understanding and dealing with the challenges of major programme management and helping business, government and civil society meet and overcome them. The purpose of the Centre is to develop leaders and scholars who understand the strengths and weaknesses of major programmes and have the knowledge, skills and tools necessary to decide when to, and not to, execute such programmes and to successfully govern, plan, manage and deliver programmes on behalf of business, government and society. See http://www.sbs.ox.ac.uk/centres/bt/Pages/default.aspx
The MSc in Major Programme Management was launched in October 2009. Informed by cutting-edge research, this part-time MSc combines the highest standards of academic rigour with a practical approach to managing large-scale, complex and transformational programmes. http://www.sbs.oxford.edu/mpm
2 About Saїd Business School
Established in 1996 the Saïd Business School is one of Europe’s youngest and most entrepreneurial business schools with a reputation for innovative business education. An integral part of Oxford University, the School embodies the academic rigour and forward thinking that has made Oxford a world leader in education and research. The School has an established reputation for research in a wide range of areas, including finance and accounting, organisational analysis, international management, strategy and operations management. The School is dedicated to developing a new generation of business leaders and entrepreneurs and conducting research not only into the nature of business, but the connections between business and the wider world. In the Financial Times European Business School ranking (Dec 2012) Saïd is ranked 12th. It is ranked number one in the UK (7th worldwide) in the FT’s combined ranking of Executive Education programmes (May 2012) and 20th in the world in the FT ranking of MBA programmes (Jan 2012). The Oxford MSc in Financial Economics is ranked 4th in the world in the 2011 FT ranking of Masters in Finance programmes (June 2012). In the UK university league tables it is ranked first of all UK universities for undergraduate business and management in The Guardian (May 2012) and has ranked first in eight of the last nine years in The Times. For more information, see http://www.sbs.ox.ac.uk/