Agility, Software Development Estimates and Buying a House

Lately, software project estimation has risen to the surface as a discussion topic at Three Wise Men. The flurry of smaller proposals we’ve done lately has driven the discussion forward.

There are several elements at play here:

1. Smaller businesses have lower tolerance to wavering budgets than larger businesses
2. Small web applications are often perceived as a commodity, like sugar or crude oil
3. Small estimates are swayed more heavily by variances caused by unseen circumstances (ie. a 1-day spike is 20% of a single-developer 1-week-sprint project, vs. 3% of a three-developer 2-week-sprint project)

See something in common here? Yup, you spotted it. “Small”.

I happen to like the word small, but sometimes businesses feel intimidated about being perceived as small. Like somehow being small is likened to being vulnerable or incapable.

Why do I like small?

Agility. Technology is an obstacle course at the best of times, a mine-field at the worst.

Communication. Just watch a meeting among 3 people versus a meeting among 30.

Focus. Fewer distractions and more personal responsibility.

Every software project should start small. With great agility, excellent communication, and sharp focus the smallest software project can change the world.

So, with this lofty goal in mind, consider the 3 challenges I mentioned above. Is there any hope for small projects?

For sure, but perhaps not where you think it might be.

For discussion, I’ll propose two ways to estimate. The first way, which seams like it should work, is what is done in many other industries.

A good friend of mine is buying a new house. It seems pretty simple - the builder has several house models they’ve built hundreds of times before. They know exactly what it’s going to take to do the framing, run the plumbing, the electrical, to shingle it. They’ve done it hundreds of times. So when they give you a price of $250,000 - that’s exactly what it’s going to be and they know it.

With software, it seems complex. Technology changes constantly. Years ago we used to joke half-heartedly and say “Hey guys, should we use the same technology on this project as we used on the last one?” - the reality was we couldn’t. The market’s drive to make more efficient and feature rich tooling and frameworks always push us forward as we strive to more wisely allocate our customers’ budgets and provide them better value. While the applications looked pretty similar on the outside, their foundations shifted constantly. So when you never build something the same way twice, because to do so would be a disservice to your customer, how can you possibly know how long it’s going to take?

Well let’s take a look at these two points in more detail.

My friend is going through right now what I’ll call the secondary pain. Up front, the house was $250,000, cut and dry. But they wanted 9’ ceilings. Oh, well, with 9’ ceilings the standard kitchen cupboards are going to look ridiculous so they need an upgrade. Oh, and you want a better shower? That’ll be another $3K. Oh, and laminate countertops aren’t your preference? Well, we have granite for somewhere between $5K and $10K more. Oh, and we realize you thought this was going to roll into your mortgage except we need 30% of your $30K in upgrades up-front. Hope you can accommodate that.

With software, you are met with tremendous initial uncertainty. Until you start accomplishing small tasks, then one by one the project falls into order. You’ll dodge and weave around challenges. As you progress along your project, you’ll see what has been accomplished and what is remaining. You’ll be able to play with it, discover new ways you can make it better. As you approach the end of your budget you will begin to more clearly see what you can accomplish and what you’re going to have to cut out until the next phase. Don’t take your eye off that budget though, it comes up fast.

You see the difference there? You have this well-established system with perceived stability up-front that entices you forward. You jump in, and then you see the sharks. Or you have this pool of cold scary-looking water you can’t see into, but once you jump in and spend some time there it doesn’t seem so cold and you begin to have some fun - but if you’re not careful hypothermia sets in.

Now I’d like to challenge one last point above. The bit about software having tremendous initial uncertainty.

If every successive custom application is built upon this shifting technology landscape, how can you possibly find order in that?

Well, to be honest I’m not sure :) but I’ll point out some recent observations I’ve made.

First, let me point out that I firmly believe software projects cannot be accurately estimated until they’re complete. The larger the project, the less accurate you can get. Thankfully by this principle, smaller projects are naturally a little easier to pin down.

I’ve always held onto a simple idea that most small software projects can be estimated using certain metrics. Once you measure out what’s involved, you should have a reasonable sense of how long it’s going to take. Of course, this doesn’t take into account new technological hurdles, or the odd developer “Eureka!” that shaves a day of coding off because they found or invented a new way to do something.

We’ve also recently begun playing Planning Poker for new project proposals. We take our interpretation of the initial project backlog and start throwing down estimates in a team setting. Often this goes pretty smoothly, but sometimes you end up with some real insightful discussion around the table.

And strangely enough, the estimates we get from both approaches tend to come out reasonably close.

I suspect there are some pretty simple answers to this actually. A strong architectural minded developer will juggle quite a bit in their head when evaluating a task. They’re going to be at least subconsciously evaluating similar metrics to those we extract methodically. What they have that our spreadsheet doesn’t though is a strong instinct for the “tough bits”. Those rough edges you just can’t pin down, because you read a blog post 3 months ago warning you about something, or maybe it just smells wrong.

And for this reason, we have begun to favour Planning Poker as a means of establishing initial budget expectations for our larger projects that go beyond simple database tables and CRUD (Create - Read - Update - Delete) manipulation of business data. In fact, for both large and small projects we’ve adopted a mix of each technique to improve our estimates.

So there you have it. Start small. Don’t be afraid of initial uncertainty. Watch the features unfold and watch your budget. 4 points to help make any software project successful.