Apr 18, 2013

Hm, A Forecasting Model. Now What?

Models are all around - model cars, model of thinking, fashion models and predictive models. We know how to use or enjoy most of them. However, forecasting models and predictions are something different and I feel there is a small confusion how to deal with them and their results. Some expect a model to tell them exactly what will be. Few of those are right as there are limited matters where a good prediction is achievable. Others totally disregard the forecasts. And rightfully due to the abundance of wrongfully placed and wrongfully communicated forecasting projects. The group of managers that use forecasting results just as something they have to have included in their reports without any consequences for future is large as well.

No matter the fancy math, thorough logic, expensive geeks and equipment involved in a model, this is still a model - a partial presentation of reality. A presentation of current reality or our understanding of it. Forecasts also are made based on the best knowledge in the organization now. No matter how fancier and new is the math or how expensive is the equipment and the geeks using it, the model is still based on what we know now. I think these are the central tenets for understanding and using forecasts.

Before making our mind about a forecasting model we should try to understand few things. Among the first to mention is the scope of the model - what exactly are the aimed outputs. It does sound trivial but one could jump to a conclusion from a few words based only on experience and context and risking to miss the real purpose. Other important topic to clarify are the foundations of the model. These are the answers to questions like what data goes in, what is the input data coverage in terms of time, categories, space and so on. I doubt you would trust a model of luxury car market in Europe for next 5 year based on the total car sales in France, Germany and Greece for the last two years. Another important part to get is the assumptions behind model. These could be the various short-cuts in input data, approximations in estimations, applied segmentation or dependencies between entities in the model. For example, a linear relation between the declining prices of mobile phones and adoption rates is not something to be trusted because of the adoption curve followed by the market of new technologies. The methods involved in the forecast are another thing to pay attention to. Often this requires some math or statistical knowledge and sounds scary but almost everyone involved in decision making has some exposure to math and could tell when something does not make sense in the employed methods. Last but not least is the performance of the model compared to a historical period a test period or to other models. I don't think there is a rule what part goes first and what next - it is very context specific. Of course, this is only when one have to make her mind about a forecast. In case the boss tells you this is a good forecast and you should use it, then use it - no need of thinking.

If the model check all the ticks, its results should not be used straight away. Forecasts tell us what would happen if things behave the way they have behaved so far in an environment as we know it. The forecast should be a basis for discussions in the organization as well as for critical review of current knowledge or any further research or analysis. Even bad forecasts could bring great benefit to the decision making process while badly used good forecasts could have detrimental effect. I see most most of the forecast as a good starting point. No matter how expensive and how extensive the model, a good portion of local and specific knowledge is not included in it. Not only is not included but also cannot be included - like government or competitors intentions that are heard over a dinner party and could mean totally different market than the one predicted by the model. There is lot of knowledge and facts that cannot be put into figures and formulas and only a human can account for it. The forecast should be the first approximation as the term goes in the mathematics. The humans adjust it to real world. A car salesman could tell you such details about this market as no model can include. Of course, human knowledge has its limitation but I would not touch on these limitations now.

I skipped the various success stories of a forecasting model that work very well to focus on the more general case.

No comments:

Post a Comment