Crystal ball

Estimates are hard... so don't do them

"Time... is what keeps everything from happening at once."
—Ray Cummings

Few things in the world can create more subtle stress for a developer than being asked to provide an estimate. Even more overt a stressor is to inevitably miss that estimate. As a species, we are not particularly good at estimates, so why do them? Just make your work continuously transparent.

[ Special Coverage: All in on All Day DevOps ]

The problem with estimates

Want an example of what's wrong with estimates? Warren Buffet recently won a 10-year old bet with hedge fund manager Ted Seides that he could not pick five hedge funds that would beat the S&P 500 index. Seides is an experienced manager, and he bet across a long period, where adjustments could generally be controlled. But even with the advantage of being able to control some of the variables, he still lost the wager. 

Hedge fund managers get paid to predict the future. Ted Seides' gamble is just one example of how humans are terrible at predicting it. People tend to get distracted or emotionally involved in the outcomes. And they are subject to a huge number of cognitive biases that impair their ability to accurately assess the limits of our capabilities and to predict outcomes.  

World Quality Report 2017-18: The state of QA and testing

Just passing the time

Typically, humans feel as though they understand the passage of time. We have devices and disciplines devoted to measuring and tracking it. And for rote and trivial behaviors, we’re actually not terrible at estimating approximations. Given some repeated behavior [B], the estimation formula probably looks like this:  

( (Total amount of time required over a lifetime to perform B) / (number of times B was performed ) ) + (whatever amount of time the chaotic nature of the universe injects into a given instance of B) 

Your daily dose of chaos

Suppose your daily commute typically takes you an average of 45 minutes. It has taken about 45 minutes a day, five days a week, 50-ish weeks a year, for the last four years. If someone asked you to be at work at 9 a.m., you’d probably leave sometime around 8 a.m. because you have experienced the chaos of traffic more than a few times and you realize that delays are possible. You have around 1,000 instances of data to assess, with some notable experiences of much more or less than 45 minutes.

But in this case, you’re not actually estimating the amount of travel time. You’re estimating the amount of chaos, a quantity that is difficult, if not impossible, to guess. Will there be an accident on my route? Will I have car trouble from my otherwise well-performing vehicle? Will aliens attack and destroy our interstate systems to prevent us from mobilizing our military? Will zombies rise? Will the interstates be empty, like the inbound lanes to Atlanta in the Walking Dead? Will I spill my coffee and have to go back home to change my trousers?

Way back when

Remembering back to those first weeks on the new job, you probably were not so sure about how long it would take to reach your destination. Most of your estimate accounted for the unknowns. You probably departed early. But over time and with repeated iterations, your estimate of the known quantities became more accurate, and your ability to assess the possibility of chaos improved marginally. No zombie uprisings, so far.

Now consider a different task: Assembling a washing machine from its constituent parts. I don’t mean a partially-assembled one. I mean every nut, bolt, bracket, motor, and panel. You have a large, unsorted pile of parts and an assembly manual for a different model that was produced five years ago. You know how washing machines should work, but have never assembled one before. Building complex software can be a lot like that—known parts producing a less well-known result.

Frequently, the resource requirements for software are mostly constrained by chaos. By chaos, I don't mean actual randomness. Simply put, the universe is a complex place with a lot of simultaneously moving parts. To know what all those parts were would require omniscience, which is in short supply. If you knew how the dice were going to land, gambling would have no risk associated with it. If you knew the actual outcomes of your project before they arose, you wouldn't be estimating. You would simply be chronicling future history.  If you could do this accurately, I expect you would be in high demand.

Writing a method that reads a file into an array is like your daily commute; you’ve done it a thousand times and understand most of the chaos. Assembling that washing machine, on the other hand, is a one-off. Completing the job successfully is totally possible given enough time and attention. But if someone asked you how long it would take you to do so, you would likely start to question their reasoning and sanity.

How to fail at software estimates: A 7-step plan

 The cycle of failure in estimating looks something like this:

  1. Speculate about the amount of work, especially time, necessary to produce X
  2. Tell someone that value, noting that the value is really mostly speculation and that there are unknown aspects to it
  3. They reassure you that this is just an estimate and you won’t be held to such unknowable arbitrary measurements in your production of X
  4. Discover that step 3 is not true by virtue of being held to the estimate
  5. Miss the estimate deadline for X
  6. Suffer
  7. Repeat from step 1

There are variants on this, of course. In one, someone tries to inject new resources that are not "extra time" into the process. In another, they ask you to produce a somewhat inferior X in order to manage the costs. In a third, the estimate gets accepted but somewhere along the way someone decries the work as "going too slowly," with the attendant histrionics.   

This cycle, the described variants, and the nigh-infinite other ways that additional stressors can be applied to the estimation/execution process frequently induce poor performance, decreased morale, and ultimately become a driver for leaving software development for a job renting Jet-Skis on the beach.

Takeaways from the #noestimates movement

Several years ago, Woody Zuill, Neil Killick, and a few other folks were aggressively promoting the #noestimates movement. At first, the logic seemed questionable. After all, I had spent decades giving estimates and performing the work around them. You give estimates; that’s just how it works. 

But consistently slipping schedules were damaging morale and reputations. So the #noestimates movement seemed like it might be at least a partial solution. The basic premise that the provision of estimates is simply giving someone a lever to manage your effectiveness and happiness remains relevant.

The nature of a resource, like time, is that it has an associated cost. Employers typically want some sort of value for that cost. So how do you get someone to agree to accept an unbounded cost for production of Feature X? You don’t. But the key part of an estimate is the boundary, and that boundary does not necessarily have to be based directly on time.  

Project managers are tasked with reporting the state of the work and dealing with the logistics of prioritization. Given that they’re not assembling the washing machine themselves, they have very little control over how that work actually transpires. 

In reality, the task of project management is to reveal the status of work to the people paying the cost for that work.  

You cannot avoid this under most circumstances.

The solution: I can see clearly now

A more effective means of revelation is to convince someone that what they really want is transparency. They really want to understand how much work remains. To provide this transparency, you must keep an up-to-date representation of the work immediately available. This might be a project plan, a series of milestones in GitHub, or some sort of burn-down Kanban board with sticky-notes and tasks. Whatever it is, it needs to be up-to-date, accurate, and accessible. 

An accurate representation allows everyone a degree of comfort because as the list changes from the inherent chaos in the system, one can see how the distance to the goal changes. Someone could stumble across an unknown error in a dependency. Someone could discover a great shortcut for parts of the work. Someone could get sick. Someone could quit. Someone could introduce a new or changed requirement.

"Hold on!" you say. "You think I should convince someone who wants a simple number in days or weeks that they actually want to see the complex chart of interdependent elements that composes the work?" 

Yes, because the first thing (a simple number) is generally a false representation of the second. It's possible that you work for someone who would rather be told a convenient untruth than to have to execute an action to get the real information. Or maybe you're more comfortable telling that false representation than with making the process more transparent.  

Show your work

In my experience, most people don't like to lie, and most people don't like to be lied to. Estimates of complex systems are lies we tell each other about how much work is left to do. So don't tell those lies. Just show your work.

For more on my approach to estimates, drop in on my presentation during the All Day DevOps online conference. Admission is free, and you can also watch my talk after the event.

World Quality Report 2017-18: The state of QA and testing

Image: Flickr

Topics: DevOps