Why is Software Project Estimation so Difficult?

As a software development company, we face the same challenge as most of our colleagues do: setting up realistic preliminary estimations.

We go to great lengths to ensure that our operation runs smoothly and that we deliver on agreed deadlines. According to what people say on Reddit and Quora - a great number of other software development companies don’t do the same.

Certainly, there is no shortage of both anecdotal and evidential examples of “failed” software projects due to missed deadlines. Thanks to that, the IT industry has developed a reputation over the years of being extremely bad at setting up realistic deadlines and delivering on them in agreed times.

In this article, we would like to dig deeper into this subject matter and try to explain why it’s so difficult for IT professionals to set up realistic delivery dates and follow through on arranged plans.

No, People are Not Just Lazy - It’s More Complicated than That

While we appreciate the new business, this type of feedback always puts our minds in fifth gear and forces us to think about why most IT dev firms struggle with the same problem.

The clients that had previous bad experience with a software company are usually jaded and ready to enforce every micromanagement technique in the book to get us to deliver on time what they paid us to do. This is not a great foundation for us to kick things off.

Unfortunately, we can’t blame the client for being extra cautious and edgy. When they reach out to a software development company to help them develop a project, most clients usually have a deadline themselves that they need to respect. It can be a product or internal deadline - it doesn’t really matter. What matters is that their success depends on the software development company’s ability to correctly set expectations and deliver the goods on agreed dates.

That being said, it’s quite easy to understand why failing to meet deadlines is an issue that most clients don’t easily forgive and why they immediately change their tune when this sort of a thing happens.

As a software development company, you need to put in as much work as needed to deliver what you promise. Bad reputation destroys all credibility for businesses and individuals that are associated with the problematic company and marks them as “unreliable partners” for life.

The Main Culprits - Why It’s So Hard to do a Proper Software Project Estimation?

The job of a software engineer is never as simple as people like to think. Almost every software engineering project comes with a certain level of uncertainty, i.e. it's often extremely difficult to precisely estimate the time required to do the work when you’re not 100 percent sure what problems you’ll encounter along the way.

For a software developer, there are two primary types of uncertainty:

• Requiremental uncertainty - knowing in advance what you need to get a certain project done

• Technical uncertainty - knowing the technical side of implementing what you envision

Under these two groups, you can assign all the common problems:

• Mythical measurement - you can’t just throw more developers at a problem and expect them to move quickly. There’s a certain learning curve that you need to anticipate before adding new people to the mix. It takes developers some time to learn the ropes and to get properly organized. Throwing more people in without a rock-solid plan and structure will just create clutter.

• Fixating on money - when the client doesn’t like the long deadline that the developer gives them, project managers usually tend to pressure the developers to come up with a shorter deadline just to keep the client engaged. The problem is, you cannot just pretend that something will take less time because somebody doesn’t like the time/cost implications.

• Lack of experience - the less experienced developers are with a specific type of project and tech, the harder it will be for them to set realistic estimations. This is where assumptions enter the game and create problems. The developers find themselves in situations where they are pressured by their management and client to have certain estimates in an uncertain environment, so they might commit to a timeline that they haven’t fully worked out yet.

• Forgetting about testing - The testing phase is often excluded from development estimates or just added onto the end almost as an afterthought or a nod to quality assurance. Testing, however, plays a much bigger role in the development process.

For most projects, this is the first time when all the components will be brought together in a system and actually tried out. It’s also the first opportunity for code to be executed in conditions other than those created by the developer.

Testing is the process of finding problems with software – the more you do it, the better the product. You need time to take everything in and test all the elements. A few days at the end of the deadline is not enough time to find all the bugs and clean your mess before the client gives it a go.

Tackling Software Project Estimation Through Organization

Extracting as much information prior to making a commitment - Every project is full of big and small obstacles to conquer. In most cases, the small ones are usually the ones that hold us back and force us to miss deadlines. At Share IT, we tend to speak to clients and stakeholders deeply about the project, all its ins and outs, before making an estimate. We ask them to provide clarity around the business goal of the software and what features it will demand as possible. The more info we get at this stage, the better we can plan timelines.

• Creating a spirit of trust and collaboration between us and the client - we are talking about a relationship where any estimates are based on a mutual understanding of the risks and uncertainties associated with any estimates. That requires a mature and informed approach to estimation by both the business sponsors and the team providing the estimates.

• Thinking about technical research - All deadlines should have a decent window of time for the developers to perform proper technical research before they start sharing estimations. The adoption of new technologies is never as easy as people think, so having a deep understanding if the team will need to learn a new thing before kicking things off is of crucial importance. We build in time for troubleshooting whenever possible.

• Estimating as a team - Inviting as many people who will work on a specific project with you to discuss the scope of work and help you come up with the estimate can be a game changer. Regardless of how experienced a team of developers, sometimes you can miss some crucial steps that can throw the whole plan off track. This is why we practice group estimates which allow us to raise additional questions and cover more bases. Talking to the team prior to making an estimate helps us see all potential ambiguities or future hang-ups in a stage where we can still modify your plan.

• Thinking about cognitive biases - These things tend to sneak in during the estimation process. There are several cognitive biases that key stakeholders need to be aware of in this stage. The first one being Hofstadter’s Law. As the famous American scholar of cognitive science’s once said: “It always takes longer than you think, even when you take into account Hofstadter’s Law”. People tend to ignore all potential negatives that could happen during the software development process and their estimates often end up being overly optimistic.

Final Takeaway About Software Project Estimation

There is a commonly used tactic of adding 20-30% of time once you’re done with the estimation. That’s how you’re certain that you covered all potential surprises down the road and respect the deadlines. The truth is, teams that have a lot of successful projects under their belt are pretty good at foreseeing everything that can go wrong.

If you’re searching for someone to handle your software development requirements, but you’re worried about the precision of time and cost estimations, have no fear: you can reach out to us at Share IT to discuss the project scope and specifications in great detail.