Capacity Models in Agile Development

A capacity model helps you to forecast how much work can get done in a given time. At Agilent’s Software and Informatics Division (SID), our capacity model helped us to more than double our productivity over a couple of years. It helped our stakeholders and customers understand what to expect from us at every release. Our capacity model was and still is the foundation of our team’s confidence and our customers‘ trust.

In Agile development, teams work in short cycles called sprints. At the end of each sprint, the team must deliver working software that a customer or stakeholder can evaluate. To that end, they work with a customer or a proxy to write "stories" that can be completed, tested, and demonstrated within a single sprint. They must also estimate how much effort the story needs so it can be sized to fit a sprint. This depends on a solid “definition of done” - the set of conditions that must be met for a story to be considered complete. A clear Definition of Done - and commitment to following it - are critical to success.

To simplify estimation and keep ourselves honest, we dedicated developers and testers to each scrum team, with a target 6 dev and 3 testers/test developers per team, plus a scrum master and a product owner each of whom typically share two scrum teams. We used a fixed two-week sprint cycle. 

Agile dogma teaches the idea that estimation can be done in dimensionless units called "story points", and that each scrum team can have its own. We needed to be able to reflect how much working capacity we had so that our stakeholders could see our limits. We learned to define a “point” of effort as being equivalent to a raw staff day. A reproducible unit of work enables aggregating the capacity of several teams into an estimate of total capacity. We delivered software to market on a fixed 6 month release cadence, and gave first priority to quality (both freedom from defects and fitness for use), second to hitting the cadence, and third to scope. The team designed releases so that scope could be ejected if things didn't go perfectly.

We found that it was critical to have a single system of record for all work and estimates (we used Atlassian Jira). This let us assign work flexibly among teams and generates useful release metrics.

Before we had a capacity model, we planned releases from a ranked backlog of work we wanted done, but found that a lot less work fit our release at the start of development than we had hoped. We were over-planning relative to what we could get done, and disappointing our stakeholders. We made our first capacity model in response to this challenge. Over time, this has resulted in reduced planning effort and much more realistic expectations about what will fit in a release, which leads to healthy give and take during estimation and story refinement.

Our first capacity model accounted for estimation errors, defect fixing time, time for customer emergencies, and time to maintain prior releases. Subtract this from “loaded” staff-days in the release and we had an estimate of capacity for stories delivered to market (“payload”), including development and testing.

Focus Factor

To estimate loaded staff-days we relied on a “focus factor” to estimate how much of a developer’s day can actually be used for developing, as opposed to other activities of corporate life. Note: Special thanks to our Agile coaches for introducing us to this idea. We treat Focus Factor as improvable and look for ways to grow it to improve the workplace for developers and testers and increase payload capacity. Note also that we assume developers work 5 day weeks and take their vacation and holidays - at least for planning purposes. No good comes from planning to burn people out.

Continuous Improvement

We refined our capacity model after each release as part of our continuous improvement efforts, using Jira to review how we spent capacity in the release. We identified un-modeled capacity leaks and set realistic goals for improving them over time. We also set goals for activities that can lead to improving focus factor. Because we value trust and autonomy, we do not measure the focus factor individually. Instead, we infer it from aggregate performance and work to improve it based on the evidence we have. 

Learnings

Over time we refined our model to include the capacity required to scale the team (training and hiring), capacity reserve for release planning, time lost to personal emergencies and attrition, and reserves to support cycle time targets for bug fixes. Experience also led us to refine our Definition of Done in response to situations where teams felt pressured to close out stories to be “on track” while pushing defects and technical debt later in the release. We added the expectation that stories have architectural review and code has peer review, reviewed unit-tests exist and pass, and that all critical and serious defects against the story are closed before a story can be marked “done”.

It’s critical to develop a culture that rewards honesty and early action when problems arise. Ultimately it is people and culture rather than processes and rules that create great products and services.

Do just enough planning. Continuous grooming and high-level rough estimation give us the evidence we need to minimize over-planning waste, especially important because Product Owner capacity is critical and difficult to scale quickly. Product Owners operate at three levels: portfolio, product, and release backlog. If there is not enough of the right kind of capacity reserved for laying the tracks (architecture and longer-term scope) the release trains will slow down.

Assess technical risk and architectural changes early in each release cycle, then work the risk down early in the release. Otherwise, you will get late surprises and it will be too late to eject scope when the crunch hits. Minimize Work in Process for the same reason - stories and features should be finishing all through the release rather than piled up at the end to level test load and reduce risk early.

Useful at any scale

Based on our experience, a capacity model can be useful at any scale, individual to enterprise, and applies to any kind of work, although it is especially good for work done with Agile methods because of their transparency. It enabled us to answer planning questions, to focus on those things that had a fighting chance of getting built in the next release, and to reflect the real costs of change and waste. With our capacity model, we confidently predicted what we could achieve, how fast we could scale, and improve every time.

Insights

Figure: Example capacity model showing a total payload of 65% and other uses including Defect fixes, Mis-estimates, Sustaining work, capacity lost to attrition and emergencies (“Cap Loss”), and planning effort.