Productivity Measurement in Agile teams

Many organizations have moved from traditional methods of project management (waterfall) toward Agile and/or DevOps approaches.

Agile software development empowers teams. It enables software to be  developed centrally, rather than with a project-like focus. This requires less management control.

The main reasons for this are:

  • it is more difficult to measure productivity due to the adoption of story points on a team level.
  • the inability of many organizations to implement a standardized method to use on a higher level for productivity measurement, productivity improvement, benchmarking, long term estimation and contract KPI’s.

This short paper explains why many organizations have ‘lost a grip’ on these crucial activities after transforming to the ‘Agile world’. At the end of the paper, as a recap a few often-heard myths are “busted”.

The Need For standards

The main problem is that the industry needs a standardized, objective, repeatable, verifiable way to measure the output of Application Development. Preferably a method that can be applied easily or that can be automated.

Until now, the international ISO/IEC standard for functional size measurement is the only useful standard to measure the output of software development in an objective, repeatable, verifiable and therefore defensible way.

IFPUG, Nesma and COSMIC methods all comply to this standard and maintain standards of their own. These methods can usually be applied easily in most Agile teams as they can measure user stories without a problem and don’t cost a lot of time or money. However, there are several reasons why the use of function points is not popular in agile teams:

  • Function Point Analysis, whether IFPUG, Nesma or COSMIC, is a very specialist expertise and many organizations don’t have knowledge to understand the value of function point analysis or the skills and expertise to carry out this type of analysis.
  • Function point analysis is perceived to be expensive, while in fact recent studies and experiences of many project managers/Scrum masters is that it’s very easy and quick to measure (COSMIC) function points based on user stories.
  • Recent studies show that estimation based on (COSMIC) function points are more accurate than studies based on story points.
  • However, Agile teams often consider function points to be an activity of the ‘old traditional world’ that should not have a place in the modern world. However, function points is a standard for the universal concept of size. Just like the meter or the foot, size is independent from the development methodology, materials used or other non-functional requirements.

As agile teams use Story points, they don’t see the need to use other measures and they have somehow convinced management that this is the way to go. However, management now lacks the ability to control the functionality delivered and they are not able to understand the team capabilities compared to the outside world.

Thus, crucial activities like productivity measurement, productivity improvement and benchmarking are not carried out any longer, which poses a big risk to many organizations.

Myths busted

Recapturing, just a few myths are busted in the next table:

Myth Truth
Velocity burn-down charts show the progress of the team in an objective way As story points is not a standard, they can be manipulated easily, especially when used in contract KPI’s. This is usually referred to as the concept of Story Point Inflation.
Function Points were suitable to measure Cobol software running on mainframe computers, back in the eighties. Function Point Analysis is an ISO/IEC standard to measure the functionality that software provides to the user, regardless the technical environment and regardless the non-functional requirements. Just like the meter was used hundreds of years ago and is still used to measure length, function points can always be used to measure the functionality provided by software to the users.
It’s expensive to use function points and the accuracy is low. The high-level variant of function points can easily be learned in one or two days by Scrum masters, requirements analysts or other team members that understand functional requirements. FPA costs less time per sprint than the sprint planning sessions and produce more accurate and reliable results.
Measuring function points should not be done in an agile team, as it is considered waste (in Lean terms). For the team, FPA may be considered waste, as story points metrics give them enough grip on performance on the team level. Management however needs grip on budgets, contracts, performance and need to make decisions on team size. As it is impossible to base this on story point metrics, FPA is definitely not waste on a management level.
Even if we have the function points, there is no data available to compare to and it will take a long time to build up metrics. The ISBSG repository has data of over 8000 Application Development projects, releases and sprints in Excel format to start with. As sprints are usually quite short (2 or 3 weeks) building up data will go quite fast.