This page is maintained by Paul Coombs, founder of IT Project Estimation Limited. I try to maintain links to information that is current, detailed and informed - there are plenty of other papers in this field that state the obvious or rework old ideas. Comments and further contributions are welcomed - mail them to references@itprojectestimation.com.
Any books referenced below are linked to the Amazon.co.uk website (because I'm British!). The books can also be found on Amazon.com using its search facilities.
The BCS Review of 2001 contains a survey of 1027 projects, of which only 12.7% were successful. There is an analysis of the reasons for failure, the stage at which failure occurred, and the skills needed in a project team.
The Standish Group “Chaos” Report is a survey of 8380 IT projects undertaken by 365 companies. Of these, 16.2% came in within time and budget. Of the successful ones, only 42% ended with the functionality originally planned. The Standish report is criticised in a paper by Magne Jørgensen and Kjetil Moløkken called How Large are Software Cost Overruns?
Software Cost Estimation with COCOMO II by Barry Boehm has a list of major IT projects and their cost overruns. Each project in the list was cancelled after expending up to four times its original budget.
Exploring the Outer Limits: How Much Software can be Developed In a Year?
by Doug Putnam of Quantitative Software Management Inc (who market the SLIM tool) contains an analysis of 6,322 completed projects (both successful and unsuccessful). Less than 10% of these managed to produce more than 75,000 lines of code in 12 months. The poor return from adding more staff to a project is demonstrated.Some other surveys are referenced in A Review of Surveys on Software Effort Estimation by Kjetil Moløkken and Magne Jørgensen.
Before you start at all, try the estimation quiz to evaluate your proficiency as a number guesser. It is a part of the book Programming Pearls by Jon Bentley.
IT Project Estimation: A Practical Guide to the Costing of Software
by Paul Coombs (Cambridge University Press, ISBN 052153285X) is an overview of all estimation methods. It concentrates on bottom-up techniques and shows how a cost model can be constructed and used. More details here.
 |
Software Estimation: Demystifying the Black Art by Steve McConnell (Microsoft Press, ISBN: 0735605351) is certainly the best book I have read on the subject (other than my own, of course!). It provides a comprehensive review of the various estimation methods and contains some wise philosophy about what an estimation is for and how to set expectations. Many of the techniques described are over-familiar to anyone who is active in this field (function points, COCOMO and so on) and some of the text is a little too self-evident to be worth mentioning. McConnell also seems keen to split "estimation" from "planning" - activities which go hand-in-hand in the world in which I live, particularly in early stages of a project when such things as function points and lines of code are not so easy to ascertain. As a result, the bottom-up method, so beloved by myself, gets relatively scant coverage in favour of more academic techniques. But overall this book is comprehensive, readable and good value for money. |
Estimating Software-Intensive Systems: Projects, Products and Processes by Richard D Stutzke (Addison Wesley, ISBN 0201703122 is a 917-page blockbuster (with CD). Much of it is self-evident or equally well covered by other references on this site, but overall this book is an impressive achievement
, if rather over-academic compared with the two rather more approachable volumes listed above. by Capers Jones (McGraw Hill, ISBN: 0079130941) is currently out of print. Capers Jones founded Software Productivity Research, who market the KnowledgePLAN tool; he has authored many other books and papers on estimation and software engineering.Making the Software Business Case: Improvement by the Numbers
by Donald Reifer (Addison Wesley, ISBN: 0201728877) looks at both sides of the coin – presenting the business case for an IT project in terms of costs and benefits. It shows how the numbers can be presented in ways that decision-makers expect and understand.The Software Productivity Consortium provides processes, methods, tools and services to its members. These include programmes for software measurement.
The International Software Benchmarking Standards Group maintains and exploits a repository of international software project metrics to help improve the management of projects through improved estimation and productivity, risk analysis and benchmarking. Their ISBSG Estimating, Benchmarking & Research CD contains metrics on more than 2,000 projects, plus two estimation tools.
The Society of Cost Estimating and Analysis
is a non-profit organization dedicated to improving cost estimating and analysis in government and industry, and enhancing the professional competence and achievements of its members. It publishes the Journal of Cost Analysis & Management, and runs courses leading to professional qualifications. Their 2003 National Conference Papers contain some material of interest.Dan Snell of Bournemouth University maintains a site (hosted by Exeter City Football Club - and why not?) that divides estimation techniques into the categories of Algorithmic Models, Expert Judgement and Machine Learning. The last of these is the most interesting, and a little-explored area.
Research being undertaken at the University of East Anglia, School of Computing Sciences is aimed at proposing ways in which public sector procurement can be done more efficiently. In particular, Use Cases are considered from the perspective of their ability to help provide effort estimates and to aid in the management of IT projects generally. The site has many links to papers concerning Use Cases and how estimates may be derived from these (see also the mention of Use Case Points below).
Several papers from the Simula Research Laboratory are relevant to this area. In particular look out for papers authored or co-authored by Magne Jørgensen, Kjetil Moløkkenor and Bente Anda.
Large Limits to Software Estimation
by J.P. Lewis of Disney TSL provides some persuasive arguments as to why objective estimation is impossible to achieve.Creating an Work Breakdown Structure
by Kim Colenso of Artemis International will help with the generation of a list of tasks to be estimated.A video entitled The Software Cost Estimation Process, introduced by Doug Brindley, senior vice president of Software Productivity Research, and featuring estimation gurus Lawrence Putnam and T. Capers Jones, is available from WatchIT.com for $195 ($495 for the CD).
The NVC Method of Software Project Estimation
is a radical new approach that takes considerably less time and is probably just as accurate as many more sophisticated techniques.
Software Cost Estimation with COCOMO II
by Barry Boehm also gives an outline of the ‘Mark I’ process.Software Sizing and Estimating: Mk II Function Point Analysis
by Charles Symons (John Wiley, ISBN: 0471929859) provides full details of Mark II Function Points.Controlling Software Projects: Management, Measurement, and Estimation
by Tom DeMarco (Prentice Hall, ISBN: 0131717111) is the book for the Bang methodology.Capers Jones’ Applied Software Measurement : Assuring Productivity and Quality (McGraw-Hill, ISBN: 0070328137) covers the calculation of Feature Points and assesses these against other Function Point metrics. This book is currently unavailable.
The definitive paper is A J Albrecht and J E Gaffney Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation. IEEE Transactions in Software Engineering, November 1983. I don't think this is available on the web.
The so-called “backfiring table”, to allow you to convert Function Points to lines of code for various languages is available to subscribers to the services offered by Software Productivity Research.
Object Points are described in An Empirical Test of Object-Based Output Measurement Metrics in a Computer Aided Software Engineering (CASE) Environment, R. Banker, R. Kauffman and R. Kumar, Journal of Management Information Systems (1994).
Estimating Software Earlier and More Accurately
by David Garmus and David Herron continues to bang the drum for Function Point Analysis.Use Case Points
There has been some recent
interest in the Use Case Points first proposed by Gustav Karner of
Objective Systems SF AB.
Estimation
Using Use-Case Points by Mel Damodaran and Aqua Netta E.
Washington from The Proceedings of ISECON 2002 summarises the
technique.
The
Estimation of Effort Based on Use Cases by John Smith of Rational
Software provides some criticism and develops the methodology further.
Improving
Estimation Practices by Applying Use Case Models by Bente Anda of
Simula Lab shows how the method
worked with some projects in a Scandinavian software development company.
Comparing
Effort Estimates Based on Use Case Points with Expert Estimates,
also by Bente Anda, reports the results from a study conducted to evaluate the method by
comparing it with expert estimates.
Duvessa
Software offer a tool to determine the Use Case Point count; it has
interfaces to Rational's applications.
On the website of Liemur Limited is a short article about use case point estimation and a free fully functional use case point estimation Java application for download.
The Software Engineering Institute has many papers on software measurement. The checklist for determining what is or is not a “line of code” is called Software Size Measurement: A Framework for Counting Source Statements by Robert E Park, and can be downloaded from this site. SEI are also responsible for the Capability Maturity Model (CMM) to evaluate the quality of an organisation’s development processes.
The Common Software Metrics International Consortium is responsible for the definition of Cosmic Full Function Points (COSMIC-FFP). Their website shows where the Measurement Manual, which defines how to calculate the metric, can be downloaded.
holds conferences, courses and workshops, and refines the counting rules. They have published a book entitled IT Measurement: Practical Advice from the Experts (ISBN 020174158X) which has contributions from Capers Jones, Howard Rubin, David Garmus, Lawrence Putnam, and Elizabeth Clark.There is an extensive bibliography of literature relating to Function Point Analysis at the website of the University of Quebec’s Software Engineering Management Research Laboratory and another at the archives of the comp.software-eng newsgroup.
Software Measurement Services offer consultancy and training in function point analysis. Their site also lists some additional papers, books and links.
The bible for COCOMO users is Software Cost Estimation with COCOMO II by Barry W Boehm et al (Prentice Hall, ISBN: 0130266922).
The definitive COCOMO website is at the University of
An adaptation of the COCOMO II estimating model called WebMo describes a new sizing metric called Web Objects. The paper is by Donald J Reifer and can be found here.
|
A useful outline of some of these tools can be found on the site of the Computer Science Department of the University of Calgary.
A review by Capers Jones of the evolution and status of software estimation tools can be seen here.
A review of the CostXpert and KnowledgePlan tools can be found in Information Week magazine.
Measures for Excellence : Reliable Software on Time, Within Budget by Lawrence H Putnam (Yourdon, ISBN 0135676940) is a description of the model behind the SLIM tool.
Software Project Cost and Schedule Estimating : Best Practices by William H Roetzheim and Reyna A Beasley (Prentice Hall; ISBN: 0136820891) describes the CostXpert tool and includes a trial copy.
A paper entitled Why Planning is Better with Estimation than Without includes some information on the ESTIMATE Professional tool.
Chemuturi Consultants offer an estimation tool called EstimatorPal (also a multi-user version called EffortTracker). These automate the processes of Function Point, Object Point, Use Case Point, and Task-Based estimation techniques using standard algorithms. See http://www.effortestimator.com
The following books provide descriptions of possible risks, and the ways in which they may be controlled: Assessment and Control of Software Risks by Capers Jones (Yourdon Press, ISBN: 0137414064), and Managing Software Quality and Business Risk by Martyn Ould (Wiley, ISBN: 047199782X).
Contingency When proposing IT Service Projects: The Supplier's Viewpoint by Mike Cooper of New England project Services provides a useful review of different types of contingency and how it may be controlled in a supplier/customer situation.
Critical Chain by Eliyahu M Goldratt (Avebury; ISBN: 0566080389) is written as a novel, albeit one where the central characters have long conversations about project management technique. The same is true of his best-known work The Goal (Gower Publishing Limited, ISBN: 0566074184), which while not directly relevant to IT projects, has insights into ways in which all processes and systems can be improved. If you prefer textbooks to novels, Project Management in the Fast Lane by Robert C Newbold (St Lucie Press, ISBN: 1574441957) is an excellent guide to Goldratt’s methods, and how they can be applied in the real world. Related websites are listed on http://www.goldratt.co.uk/links.html.
The Wideband-Delphi review method is described in Software Engineering Economics by Barry Boehm (Prentice Hall, ISBN: 0-13-822122-7). A summary can also be found in an article entitled Stop Promising Miracles by Karl E Weigers. Peer Reviews in Software: A Practical Guide (Addison Wesley, ISBN: 0201734850) by the same author covers the review process throughout the development lifecycle.
Information on Microsoft Project is obtainable from the Microsoft website.
How to Defend an Unpopular Schedule by Steve McConnell of Construx Software explores the psychology behind negotiations between developers and management in order to agree a schedules.
Reducing Bias in Software Project Estimates by David Peters and George Dewey describes some management techniques to obtain estimates free of personal bias.
Computer Economics is an independent research organization that specializes in providing economic and strategic analysis and data to IT and business executives. The data they collect regarding IT spending may be of use in determining the cost/benefit of proposed projects or the calculation of a daily rate for each project role.
Earned
Value Project Management
by Quentin W.
Fleming and Joel M. Koppelman (Project Management Institute; ISBN:
1880410273) is a brief, readable but comprehensive guide to this
technique.
Earned Value Management Information on earned value project management for government, industry and academic users maintained by the Office of Force Transformation of the Department of Defense.
Earned Value Papers and FAQs about Work Scope Definition, Scheduling, Resource Allocation and Budgeting, Status Monitoring, Management Reporting, Corrective Action, etc. Maintained by Noel N Harroff.
College of Performance Management assists the earned value professional and project manager in professional growth and promotes the application of earned value management.
The Simula Reseach Laboratory maintains BESTweb (BEST = Better Estimation of Software development Tasks). It is a valuable source of papers about estimation and related subjects, and certainly worth investigation. To do so, you can use my user id (user: Paul, password: PaulCoombs, which is case-sensitive) but if you wish to use the library further please apply for your own id, so they can keep track of their user base.
RS Pressman and Associates maintain one of the web's most comprehensive collections of software engineering resources. The section on project planning and estimation can be found here.
Software-engineer.org links to an excellent selection of estimation papers.
The Data & Analysis
Relevant
references can also be found at the website of the
Anything written by Tom Demarco is entertaining and insightful. In particular, Why Does Software Cost So Much? : And Other Puzzles of the Information Age (Dorset House; ISBN: 093263334X) delves into the sociological and psychological drivers behind IT projects. I don’t know how to incorporate them into your estimates, but such factors as fear of failure, losing face, rigid rules, and organisational traditions are fundamental to the success of any team, and any project.
The Mythical Man-Month by Frederick P Brooks (Addison Wesley, ISBN: 0201835959) references projects from a long gone age, although the latest edition has been revised with new thoughts from the author. However its fundamental points are still as applicable as ever – the last 10% of a software project may take more resources than all used so far, and adding resources to a failing project will only make it finish even later.
All books by Douglas R Hofstadter are inspirational and challenging. Specifically, his essay On Number Numbness discusses just how hard it is to envisage what large numbers really mean. It can be found in Metamagical Themas : Questing for the Essence of Mind and Pattern (Basic Books, ISBN: 0465045669).
And finally, Edward Yourdon’s Deathmarch (Prentice Hall; ISBN: 013143635X) covers the dark side of the estimation world - impossible projects. It discusses how such projects arise and what to do about it when doom is inevitable.
