Need an Estimate? Borrow My Crystal Ball

Project Manager: How many software engineers does it take to screw in a light bulb?

Software Engineer: I don’t know—estimation is impossible.

Have you ever run into this? The punchline has a couple of things going on.

One is the utter humorlessness of not being able to recognize a joke.  I am not talking about that today.  We will leave that mystery of life to the cognitive scientists.

The other is the actual joke, that many software engineers will refuse to estimate even something as seemingly easy to know as how many people are needed to complete a simple job.  Or given one person, how much time is needed to complete a simple task. Easy…simple…these are words that have been used to sucker them in the past.  Something that is easy should not take more than what, a couple of minutes?  Hours?  Days?

This is the land of blatant speculation and guesswork, and any software engineer who has fallen into the self-made trap of committing to an impossible deadline based on their own prideful estimates is loathe to repeat the mistake.  Let me explain.

Eager, smart, self-respecting, shy, the typical software engineer wants to appear at least as competent as the other engineers in the area.  Asking questions is awkward, since having to ask looks like not knowing, which surely is a sign of ignorance, which is definitely on the wrong side of competence.  The smartest thing might be to think of all of the details based on past experience that have not been supplied to uncover all of things that can go wrong that will take additional time to prevent from happening in the first place.  Only in that way can the answer to the question “how long” include every contingency to cover for the lack of planning the person asking the question has failed to do.  Huh.

At least that is how the internal monologue goes.  No one gets a chance to answer the questions that are never asked, to explain the requirements in a bit more detail, to consider contingency plans for something that seemed so trivial.

This particular breakdown in communication leads to one of two outcomes.  One is an overly cautious, extra large estimate for something that would be completely over engineered, with features and safeguards that are unnecessary for the intended purpose.  The other is to not answer because [insert any logical-sounding reason here, such as “estimation is impossible”].  It is easier on the ego to denounce estimates entirely and be written off as a curmudgeon.  Garrumph.

There is another form of breakdown with a software engineer who is either less experienced than his comrade in the above example or who otherwise lacks the competence or care to make a reasonable estimate but still wants to look good.  Also without asking questions, the answer will come back as a single, almost miraculous number, without any qualifications or clarification.  “Ten days,” or whatever.

Later when the time has blown past and the project is officially late, the engineer just shrugs and says he is 90% done…every day until the work is reassigned or the project is canceled.

Or maybe he squeaks something out and you go to production, where the customers gets the joy of discovering all of the problems the engineer did not find or forgot to mention.  It worked on his machine.  No one said it had to keep out hackers.  It had to look good on Safari, too?  Ugh.

At this point, it seems like I am picking on the nerdy software engineer, which might be okay because I am one of them, but that is only one side of the story.  On the other side is the person quite innocently asking the question.  Chances are that this person is responsible for deadlines and delivering the software “on time,” and that being on time is more a function of what some uninformed salesperson committed to in order to make a deal.

Project Manager: How long will it take to deliver feature X?

Software Engineer: Hmm, I have made a plan that shows the sub-features of X and how long each of those will take.  Adding some time for unit tests, system testing and a few roll-out scripts, if we pull a few late nights and a couple of weekend, we can be ready in a month.

Project Manager: Too bad, the contract says you have two weeks.

And so it goes.

In my experience, the fundamental problem with estimation is that people are usually asking for and trying to make predictions rather than estimates.  Estimates may be one essential ingredient to making a prediction, but treating them as the same thing is the source of all kinds of scheduling and commitment trouble.

Fear of being wrong and of being held accountable for failing to deliver causes breakdown in communication.  When a software engineer says “estimation is impossible,” what he (or she) means is that making 100% accurate predictions every time based on the scanty information available in a typical request for an “estimate” is impossible.

So what is an estimate if not a prediction?  A good estimate is presented as a range with a level of confidence.

For instance, let’s say I can run the 400 meter dash in about 60 seconds.  An estimate of how I will do in my next race might look like: 58 to 61 seconds with 95% confidence.  That means I am estimating a 5% chance that I will take longer than 61 seconds or be faster than 58 seconds.  Why?  Maybe one out of 20 races I will have pulled a muscle, or have to run against a strong wind, or be coming down with a cold.  Maybe I will have the right combination of energy snack, strong competition, and positive mental attitude to beat my own personal record.  Clearly, lots of factors cause a variance in the time it takes to complete a lap.

A good estimate includes assumptions that form the basis for the estimate.  The way to come up with assumptions is to ask lots of clarifying questions.  Answers to those questions become the assumptions.  If any of the assumptions turns out to be wrong, the estimate would likely need to be changed.

Take changing a light bulb.  We make a few assumptions—replacement bulbs are available, the bulb socket is within reach, the door to the room is unlocked or we have a key, the power is on, we will not have to take an urgent phone call, we are not in the middle of brain surgery, and so on.  If you wanted me to screw in that light bulb, but I am on the operating table with my head open, you might need to hire someone else.  Hiring involves a lot of people, so my estimate is going way up.

Plenty more can be said about making reasonable estimates and how those should feed into those much sought after predictions.  Ample material for future posts.

For now, I leave you with a final look at our proverbial light bulb situation.

Project Manager: How many software engineers does it take to screw in a light bulb?

Software Engineer: Is it for a desk lamp or the ceiling?  Do we have a spare bulb handy, or do I have to buy one?

Project Manager: No no, I mean you’re going to have to program a talking light bulb into our office productivity suite that will use artificial intelligence to answer questions that customers have about how to use a spreadsheet.  When the bulb is ready to answer, it will spin around like it’s screwing itself in.  See?

I predict a spirited, if not entirely rational, discussion about estimating is about to begin.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s