On productivity & predictability in software engineering
“I am completely satisfied with how fast our engineering team delivers and how well they hit the deadlines,” said no CEO ever.
We all want to deliver faster and hit those upfront estimates or deadlines. But is that a goal that we want to impose on engineering teams? Is that a goal that benefits the business as a whole? To understand this issue we need to look at it in terms of cost / benefit analysis and we need to delineate the difference between productivity and predictability.
Productivity vs. predictability
Productivity is simply how fast can a team deliver. Predictability is how accurately the team can estimate project timelines. While productivity is largely a factor of team members individual skills and the overall team cohesion (and team size), predictability more so depends on external factors. Software engineering is inherently unpredictable due to a multitude of possible unforeseen technical issues and requirement gaps discovered in the middle of project implementation. Sales, marketing, contract negotiation, and other business functions are similarly unpredictable — there are too many external factors that are impossible to account for upfront. If we really think about it, life in general is quite unpredictable.
But of course a business cannot operate without some degree of predictability. Instead of looking at it as a binary — can we hit deadlines or not — we need to look at it as a range of how much we are willing to pay for an extra degree of certainty.
So what is the cost of predictability? Unfortunately the cost of predictability is productivity. For a project to have more predictable timeline it needs to be discussed, designed, prototyped, and researched extensively before an estimate can be produced. That work takes away from the current project being worked on. It is tempting to think that all this upfront work needs to be done for the new project anyway and as such goes towards its eventual implementation, however this is only partially true. While some of this work is reusable, some inadvertently is not. The biggest hit, however, comes from context switching the engineering team, which happens twice. Once, as the team switches away from the current project and another time switching back to the current project. Moreover, by the time the new project starts many findings from research have faded from memory and need to be re-learned causing more wasted time.
The good news is that if we graphed time investment (i.e., productivity) vs. predictability it would look something like the graph on the left. What that means in practice is that we can get a good level of predictability for a moderate time investment, however if we want precise dates it will require a larger time investment and a larger hit to productivity.
Roadmapping & project management
Now that we understand the inverse relationship between productivity and predictability we can make informed business decisions. Some projects (and some business domains) require more predictability (e.g., contractual obligations, date related business opportunities, etc), however for most e-commerce businesses a week or two of difference on individual project delivery don’t matter. What matters is how quickly on average projects are delivered (e.g., how many projects per year a team can complete).
What we typically want to optimize for is quicker average delivery with acceptable predictability. A mature software team can predict projects with 50% accuracy with minimal time investment while still focusing on the project at hand. For those few projects that do require hard dates, we can make exceptions ahead of time and plan on doing more research and design.
With this approach in mind, the engineering team can move fast with the acceptable level of predictability. Some projects are on time and some take a week or two longer — not a big deal for each project— but over time the weeks accumulate and we soon realize that there is no way we will get to the last 2–3 projects on our yearly roadmap. And of course demanding more predictability will only backfire.
So what do we do? Take a look at another article titled “scope & on time delivery”.