Posted by George Huhn on Mon, May 17, 2010 @ 02:27 PM
What does it mean when a meteorologist says "the chance of rain today is 60%?"
Each day in the United States, a massive amount of data is collected from weather stations, satellites, and weather balloons from around the world and sent to the National Meteorological Center near Washington, D.C. The data is processed to give a multi-dimensional picture of global atmospheric conditions, and then it is analyzed using various algorithms to develop local weather forecasts and predictions.
But this isn't how they make the "percent chance of precipitation" predictions. Even with the massive amount of data and super computer speed, their predictive algorithms alone just aren't good enough. So they use comparisons to historical data.
Basically, they take the current atmospheric conditions and compare them with days in the past that had very similar conditions. So when they say that "the chance of rain today is 60%," it means that it rained on 60% of the days in the comparison set.
And guess what? Assuming the data was entered properly, these predictions are 100% reliable all the time. Why? Because they are only predictions of probability – they aren't "wrong" on a particular day, whether it rains or not. But whether they are accurate or not in the long term is an entirely different question.
The only way to determine if the predictions are accurate is to collect the data and plot the actual versus the predicted conditions over time to learn the margin of error. If it only rained on 30% of the days that the prediction was 60%, then there is a problem with the data or the data processing.
You can do the same type of probability prediction and testing with your business projects, too. The more accurate your estimates, the more confidence you will have in your overall project-value ranking in your project portfolios.
Developing more accurate project risk estimates requires 4 basic activities:
1) Identifying the key drivers of cost, time, and resource risks in completing project tasks.
2) Preparing a database of these tasks that includes the corresponding cost, time, and resource estimates assigned to each project and the basis for those estimates at the beginning of the project.
3) Tracking the actual costs, times, and resources used performing the task as each task is completed.
4) Comparing the actual costs, times, and resources with the starting estimates.
After you have maintained this database for a period of time, you will be able to plot the actual versus the predicted results. This plot will show you the accuracy of your cost, time, and resource estimates as well as revealing the distribution of the actual results. (You will probably learn that your cost estimates were too low, your time estimates were too short, and your resource estimates were for too few. And that is a good thing to learn.) Eventually, you will be able to use the actual results data as a basis for future probability predictions, including understanding the uncertainty in those estimates.
I saw the data of one major pharmaceutical company who did this for their project "percent probability of success" estimates. The data between 20 and 85% was surprisingly linear; for example, about 50% of the projects that had "percent probability of success estimates" of 50% were ultimately successful. It also showed that all projects that had an estimated "percent probability of success" of 85% or greater succeeded and all that had an estimate of 20% or less failed.
If you’re involved in project portfolio management and you're looking for ways to improve your project planning, compiling and analyzing your historical data is a great way to test and improve your future estimates.
Does your company track and analyze historical project management data? Why do you think that most businesses don't?
What are the best uses of your company's dollars and resources? Optsee
® can tell you. Optsee
® is a project portfolio management and budgeting optimization tool unlike any that you've ever seen.
Click here to find out more.
Posted by George Huhn on Fri, Apr 30, 2010 @ 10:38 AM
This last paragraph caught my attention:
Karl Popper, the great philosopher of science, once divided the world into two categories: clocks and clouds. Clocks are neat, orderly systems that can be solved through reduction; clouds are an epistemic mess, “highly irregular, disorderly, and more or less unpredictable.” The mistake of modern science is to pretend that everything is a clock, which is why we get seduced again and again by the false promises of brain scanners and gene sequencers. We want to believe we will understand nature if we find the exact right tool to cut its joints. But that approach is doomed to failure. We live in a universe not of clocks but of clouds.1
Our businesses are mixtures of clocks and clouds, but it is often difficult to distinguish which parts are clouds and which parts are clocks. The clocks are things that we can measure and control but clouds are things that we can only try to predict.
Confusion begins when we try to measure and control clouds as if they are clocks or we focus only on trying to control clocks without trying to predict and mange the clouds.
Like Leher's comment on science, it is also a mistake of modern business to try to pretend everything is a clock. While I am a strong proponent of using business analytics, such as
simulation and
optimization in
project portfolio management tools, the analytical tools that we use should be able to manage data from both clocks and clouds. And when we're using these tools, we should be ever mindful of the inherent uncertainty (cloudiness!) in the data and the resulting predictions. Business analyses should rarely be
chiseled in stone. Too often, they are treated like they are.
Look around your business today. Where are your clocks? Where are your clouds?
What are the best uses of your company's dollars and resources? Optsee
® can tell you. Optsee
® is a project portfolio management and budgeting optimization tool unlike any that you've ever seen.
Click here to find out more.
Posted by George Huhn on Thu, Dec 10, 2009 @ 02:02 PM
Good project execution is essential to achieving the strategic goals of a company, but most companies either don't measure it or don't measure it well.
In companies where the quality of project management execution is assessed at all, it is usually measured against meeting budget, timing, and resource objectives that were often ill-conceived to start with. So when projects go over budget or are under resourced or when timelines are missed, it is too often blamed on "execution," and rarely on the poor quality of the initial budget, timing, resource, and risk assessments (or lack thereof).
How often have you seen managers record a quantitative basis for their planning estimates at the beginning of a project and then assess them at the end of a project?
Too often managers look at Gantt charts as if they are THE PROJECT PLAN carved in stone. They aren't. Most of the time, Gantt charts represent the best guesses of well-intentioned people who tend to underestimate risks, resources, and timing (because that is what human beings tend to do). Most of the time, the "data" used to support the project planning either doesn't exist, hasn't been checked, or hasn't been derived empirically. Task start and end dates are fixed with virtually no meaningful or quantitative discussion about the probabilities of meeting those dates or modeling the dramatic cumulative effects of small amounts of slippage on project value.
A big contributor to those results may not be just poor project execution, but how and where the project finish lines were drawn at the start of the project. So it is past the time for managers to be measuring the quality of project execution based on chiseled-in-stone Gantt charts.
Instead, today's managers need to start using quantitative risk analyses, databases of empirical data from previous experiences, and statistical tools for probabilistic project planning so they can truly assess and improve the quality of project execution.
What are the best uses of your company's dollars and resources? Optsee
® can tell you. Optsee
® is a project portfolio management and budgeting optimization tool unlike any that you've ever seen.
Click here to find out more.
Posted by George Huhn on Sat, Dec 05, 2009 @ 03:54 PM
In spite of the fact that counting, numbers, and games of chance have been around for millenniums, it wasn't until the middle of the 17th century when three Frenchmen, Blaise Pascal, Pierre de Fermat, and Chevalier de Méré, developed the foundations for modern probability theory.
So even though probability theory seems obvious to us today, it was relatively late in human history that it was developed. With that in mind, I have been enjoying teaching some of the concepts of probability to my 9-year old son using a fun little book called
Do You Wanna Bet?: Your Chance to Find Out About Probability By Jean Cushman and Martha Weston. It is an entertaining story of two boys, Danny and Brian, discovering the impact of probability in their everyday experiences and developing a practical understanding of how to apply these concepts to others.
Oliver Wendell Holmes once said, "One's mind, once stretched by a new idea, never regains its original dimensions." Probability is one of those great ideas that can stretch a person's mind, particularly a child's. Once a child has learned the basic concepts of arithmetic, it is probably never too early begin teaching these great ideas, and this book is a wonderful way to start.
What are the best uses of your company's dollars and resources? Optsee
® can tell you. Optsee
® is a project portfolio management and budgeting optimization tool unlike any that you've ever seen.
Click here to find out more.