Project teams had, over two years, been expressing an opinion that a new way of working was inefficient. IT management and the business only reacted when the fact of a 500% cost increase was proven and presented.
One of the main banks in the Netherlands was struggling with a major estimation problem.
The software realization part of a very big project had to be estimated and there was no history data about such projects available in the organization. In fact, there was no history data at all available. However, as the financial sector was in trouble, the bank had to be very cautious about the estimation of the project. Failure would mean a large fiasco in the press and would certainly result in big image problems for the organization. Expert estimation resulted in a rough estimate of 70.000 hours, but the management felt the estimate should be based on data instead of feelings.
The first thing we did was trying to measure the functional specifications with NESMA function point analysis. We used our in-house developed method for rating the documentation and this resulted in a low figure: 2 out of 10. The requirements were not complete and not stable at that time, but still the organization demanded an estimate in order to see whether the business case could be met. The functional size was measured using the so-called Dutch method, resulting in about 3.500 function points. The methodical uncertainty margin for this method is -15% till +50%, meaning that there is a 70% chance that the product size will fall between 2.975 function points and 5.250 function points.
The next step was to try to identify a realistic productivity rate for this project. From the project data listed in the “Development & Enhancement Repository” of the International Software Benchmarking Standards Group (ISBSG), a selection has been made of projects that are comparable to the project. The selected projects show the following spread of productivity figures:
|Number of projects||48|
Based on these figures, the following productivity rates were applied for the project:
- Minimal (P10): 18,4 hours/FP
- Likely (median): 29,1 hours/FP
- Maximal (P90): 39,4 hours/FP
In the end we were able to deliver a rough estimate for the client:
|Scenario||Size (FP)||PDR TOTAL in hours/FP||Total effort|
These results have a high degree of uncertainty which ranges from -20% to +35%.
Because the ISBSG data is considered to be ‘best-in-class’ and because there were some project characteristics that were expected to be of great risk, such as the fact that there were multiple development teams in different countries who had to cooperate together, we expected the productivity of the organization to be less than average. Based on this analysis the client chose to estimate the project to be 150.000 hours. The estimate was refined after high level design. After project completion it turned out that about 138.000 hours were spent. If the original expert estimate of 70.000 hours had been used, there would have been an overrun of almost 100%.
The ISBSG data really helped this organization to estimate their project in a realistic way, preventing them for a huge failure.
Note: This is a real case, but the data is altered somewhat to ensure the anonymity of the organization.