What does it mean when a meteorologist says “the chance of rain today is 60%?”

Each day in the United States, a massive amount of data is collected from weather stations, satellites, and weather balloons from around the world and sent to the National Meteorological Center near Washington, D.C. The data is processed to give a multi-dimensional picture of global atmospheric conditions, and then it is analyzed using various algorithms to develop local weather forecasts and predictions.

But this isn’t how they make the “percent chance of precipitation” predictions. Even with the massive amount of data and super computer speed, their predictive algorithms alone just aren’t good enough. So they use comparisons to historical data.

Basically, they take the current atmospheric conditions and compare them with days in the past that had very similar conditions. So when they say that “the chance of rain today is 60%,” it means that it rained on 60% of the days in the comparison set.

And guess what? Assuming the data was entered properly, these predictions are 100% reliable all the time. Why? Because they are only predictions of probability – they aren’t “wrong” on a particular day, whether it rains or not. But whether they are accurate or not in the long term is an entirely different question.

The only way to determine if the predictions are accurate is to collect the data and plot the actual versus the predicted conditions over time to learn the margin of error. If it only rained on 30% of the days that the prediction was 60%, then there is a problem with the data or the data processing.

You can do the same type of probability prediction and testing with your business projects, too. The more accurate your estimates, the more confidence you will have in your overall project-value ranking in your project portfolios.

Developing more accurate project risk estimates requires 4 basic activities:

1) Identifying the key drivers of cost, time, and resource risks in completing project tasks.
2) Preparing a database of these tasks that includes the corresponding cost, time, and resource estimates assigned to each project and the basis for those estimates at the beginning of the project.
3) Tracking the actual costs, times, and resources used performing the task as each task is completed.
4) Comparing the actual costs, times, and resources with the starting estimates.

After you have maintained this database for a period of time, you will be able to plot the actual versus the predicted results. This plot will show you the accuracy of your cost, time, and resource estimates as well as revealing the distribution of the actual results. (You will probably learn that your cost estimates were too low, your time estimates were too short, and your resource estimates were for too few. And that is a good thing to learn.) Eventually, you will be able to use the actual results data as a basis for future probability predictions, including understanding the uncertainty in those estimates.

I saw the data of one major pharmaceutical company who did this for their project “percent probability of success” estimates. The data between 20 and 85% was surprisingly linear; for example, about 50% of the projects that had “percent probability of success estimates” of 50% were ultimately successful. It also showed that all projects that had an estimated “percent probability of success” of 85% or greater succeeded and all that had an estimate of 20% or less failed.

If you’re involved in project portfolio management and you’re looking for ways to improve your project planning, compiling and analyzing your historical data is a great way to test and improve your future estimates.

Does your company track and analyze historical project management data? Why do you think that most businesses don’t?

What are the best uses of your company’s dollars and resources? Optsee^® can tell you. Optsee^® is a project portfolio management and budgeting optimization tool unlike any that you’ve ever seen. Click here to find out more