Published in The IEBM Encyclopedia of Marketing, Michael J. Baker (Ed.), International Thompson Business Press, 1999, p. 278-290. 1. Forecasting methods: an overview 2. Direct extrapolation of sales 3. Causal approaches to sales forecasting 4. New product forecasting 5. Evaluating and selecting methods 6. Estimating prediction intervals 7. Implementation 8. Conclusions Overview Interesting and difficult sales forecasting problems are common. Will the 1998 Volkswagen Beetle be a success? Will the Philadelphia Convention Hall be profitable? How will our major competitors respond if we raise the price of our product by 10 per cent? What if we cut advertising by 20 per cent? Sales forecasting involves predicting the amount people will purchase, given the product features and the conditions of the sale. Sales forecasts help investors make decisions about investments in new ventures. They are vital to the efficient operation of the firm and can aid managers on such decisions as the size of a plant to build, the amount of inventory to carry, the number of workers ,to hire, the amount of advertising to place, the proper price to charge, and the salaries to pay salespeople. Profitability depends on (1) having a relatively accurate forecast of sales and costs; (2) assessing the confidence one can place in the forecast; and (3) properly using the forecast in the plan. Marketing practitioners believe that sales forecasting is important. In Dalrymple”s (1975) survey of marketing executives in US companies, 93 per cent said that sales forecasting was “one of the most critical” or “a very important aspect of their company”s success.” Furthermore, formal marketing plans are often supported by forecasts (Dalrymple 1987). Given its importance to the profitability of the firm, it is surprising that basic marketing texts devote so little space to the topic. Armstrong, Brodie and Mclntyre (1987), in a content analysis of 53 marketing textbooks, fou9nd that forecasting was mentioned on less than 1 per cent of the pages. Research on forecasting has produced useful findings. These findings are summarized in the Forecasting Principles Project, which is described on the website forecastingprinciples.com. This entry draws upon that project in summarizing guidelines for sales forecasting. These forecasting guidelines should be of particular interest because few firms use them. I also describe some commonly used approaches that are detrimental to sales forecasting. After a brief overview of forecasting methods, I discuss the direct extrapolation of sales data, either through statistical data or simply judgmental. Next, I describe causal approaches to sales forecasting. Attention is then given to new product forecasting. This is followed by a discussion of how to select appropriate methods and by a description of methods to assess uncertainty. I conclude with suggestions for gaining acceptance of forecasting methods and of forecasts. 2 1. Forecasting methods: an overview Forecasting involves methods that derive primarily from judgmental sources versus those from statistical sources. These methods and their relationships are shown in the flow chart in Figure 1. Judgment and statistical procedures are often used together, and since 1985, much research has examined the integration of statistical and judgmental forecasts (Armstrong and Collopy 1998b). Going down the figure, there is an increasing amount of integration between judgmental and statistical procedures. A brief description of the methods is provided here. Makridakis, Wheelwright and Hyndman (1998) provide details on how to apply many of these methods. Econometric models Data mining Statistical Univariate Theorybased Databased Extrapolation models Multivariate Rule-based forecasting Unaided judgment Judgmental Others Self Role Playing (Simulated interaction) Role No role Conjoint analysis Knowledge Source Quantitative analogies Unstructured Structured Group Individual Prediction markets Delphi Decomposition Structured analogies Neural nets Expert Systems Intentions/ expectations Judgmental bootstrapping Figure 1. Characteristics of forecasting methods and their relationships (NOTE: This figure was redesigned in Sept. 2004, as above) Intentions studies ask people to predict how they would behave in various situations. This method is widely used and it is especially important where one does not have sales data, such as for new product forecasts. A person”s role may be a dominant factor in some situations, such as in predicting how someone would behave in a job related situation. Role-playing is useful for making forecasts of the behavior of individuals who are interacting with others, and especially in situations involving conflict. Another way to make forecasts is to ask experts to predict how others will behave in given situations. The accuracy of expert forecasts can be improved through the use of structured methods, such as the Delphi procedure. Delphi is an iterative survey procedure in which experts provide forecasts for a problem, receive anonymous feedback on the forecasts made by other experts, and then make another forecast. For a summary of the evidence on the accuracy of Delphi versus unstructured judgment, see Rowe and Wright (1999). One principle is that experts” forecasts should generally be independent of one another. Focus groups always violate this principle. As a result, they should not be used in forecasting. Intentions can be explained by relating the “predictions” to various factors that describe the situation. By asking consumers to state their intentions to purchase for a variety of different situations, it is possible to infer how the factors relate to intended sales. This is often done by regressing their intentions against the factors, a procedure known as “conjoint analysis.” 3 As with conjoint analysis, one can develop a model of the expert. This approach, judgmental bootstrapping, converts subjective judgments into objective procedures. Experts are asked to make a series of predictions. For example, they could make forecasts for the next year”s sales in geographical regions. This process is then converted to a set of rules by regressing the forecasts against the information used by the forecaster. Once developed, judgmental bootstrapping models offer a low-cost procedure for making forecasts. They almost always provide an improvement in accuracy in comparison to judgmental forecasts, although these improvements are typically modest (Armstrong 1999). Extrapolation methods use only historical data on the series of interest. The most popular and cost effective of these methods are based on exponential smoothing, which implements the useful principle that the more recent data are weighted more heavily. Another principle for extrapolation is to use long time-series when developing a forecasting model. Yet, Focus Forecasting, one of the most widely-used time-series methods in business firms, does not do this. As a result, its forecasts are inaccurate (Gardner and Anderson 1997). Still another principle for extrapolation is to use reliable data. The existence of retail scanner data means that reliable data can be obtained for existing products. Scanner data are detailed, accurate, timely and inexpensive. As a result, the accuracy of the forecasts should improve, especially because of the reduction in the error of assessing the current status. Not knowing where you are starting from has often been a major source of error in predicting where you will wind up. Scanner data are also expected to provide early identification of trends. Empirical studies have led to the conclusion that relatively simple extrapolation methods perform as well as more complex methods. For example, the Box-Jerkins procedure, one of the more complex approaches, has produced no measurable gains in forecast accuracy relative to simpler procedures (Makridakis et al. 1984; Armstrong 1985). Although distressing to statisticians, this finding should be welcome to managers. Quantitative extrapolation methods make no use of managements” knowledge of the series. They assume that the causal forces that have affected a historical series will continue over the forecast horizon. The latter assumption is sometimes false. When the causal forces are contrary to the trend in the historical series, forecast errors tend to be large (Armstrong and Collopy 1993). While such problems may occur only in a small minority of cases in sales forecasting, their effects can be disastrous. One useful guideline is that trends should be extrapolated only when they coincide with managements” prior expectations. Judgmental extrapolations are preferable to quantitative extrapolations when there have been large recent changes in the sales level and where there is relevant knowledge about the item to be forecast (Armstrong and Collopy 1998b). Quantitative extrapolations have an advantage over judgmental methods when the large (Armstrong 1985, 393-401). More important than these small gains in accuracy, however, is that the quantitative methods are often less expensive. When one has thousands of forecasts to make every month, the use of judgment is seldom cost effective. Experts can identify analogous situations. Extrapolation of results from these situations can be used to predict for the situation that is of interest. For example, to assess the loss in sales when the patent protection for a drug is removed, one might examine the results for previous drugs. Incidentally, the first year loss is substantial. Rule-based forecasting integrates judgmental knowledge about the domain. Rule-based forecasting is a type of expert system that is limited to statistical time series. Its primary advantage is that it incorporates the manger”s knowledge in an inexpensive way. 4 Expert systems use the rules of experts. In addition, they typically draw upon empirical studies of relationships that come from econometric models. Expert opinion, conjoint analysis, bootstrapping and econometric models can aid in the development of expert systems. Despite an immense amount of research effort, there is little evidence that multivariate time-series provide any benefits to forecasting. As a result, these methods are not discussed here. Econometric models use data to estimate the parameters of a model given various constraints. When possible. Which is nearly always in management problems, one can draw upon prior research to determine the direction, functional form, and magnitude of relationships. In addition, they can integrate expert opinion, such as that from a judgmental bootstrapping model. Estimates of relationships can then be updated by using time-series or cross-sectional data. Here again, reliable data are needed. Scanner data can provide data from low-cost field experiments where key features such as advertising or price are varied to assess how they affect sales. The outcomes of such experiments can contribute to the estimation of relationships. Econometric models can also use inputs from conjoint models. Econometric models allow for extensive integration of judgmental planning and decision making. They can incorporate the effects of marketing mix variables as well as variables representing key aspects of the market and the environment. Econometric methods are appropriate when one needs to forecast what will happen using different assumptions about the environment or different strategies. Econometric methods are most useful when (1) strong causal relationships with sales are expected; (2) these causal relationships can be estimated; (3) large changes are expected to occur in the causal variables over the forecast horizon; and (4) these changes in the causal variables can be forecast or controlled, especially with respect to their direction. If any of these conditions does not hold (which is typical for short-range sales forecasts), then econometric methods should not be expected to improve accuracy. 2. Direct extrapolation of sales If one does not have substantial amounts of sales data; it may be preferable to make judgmental extrapolations. This assumes that the person has good knowledge about the product. For example, the characteristics of the product and market and future plans are all well-known. When one has ample sales data, it is often sufficient merely to extrapolate the trend. Extrapolation of the historical sales trend is common in firms (Mentzer and Kahn 1995). Extrapolation methods are used for short-term forecasts of demand for inventory and production decisions. When the data are for time intervals shorter than a year, it is generally advisable to use seasonal adjustments, given sufficient data. Seasonal adjustments typically represent the most important way to improve the accuracy of extrapolation. Dalrymple”s (1987) survey results were consistent with the principle that the use of seasonal factors reduces the forecast error. Seasonal adjustments which also led to substantial improvements in accuracy were found in the large-scale study of time series by Makridakis et al. (1984). If the historical series involve much uncertainty, the forecaster should use relatively simple models. Uncertainty in this case can be assessed by examining the variability about the long-term trend line. Schnaars (1984) presented evidence that the naïve forecast was one or me most accurate procedures for industry sales forecasts. Uncertainty also calls for conservative forecasts. Being conservative means to stay near the historical average. Thus, it often helps to dampen the trend as the horizon increases (see Gardner and McKenzie 1985 for a description of one such procedure and for evidence of its effectiveness). 5 One of the key issues in the extrapolation of sales is whether to use top-down or bottom-up approaches. By starting at the top (say the market for automobiles), and then allocating the forecast among the elements (e.g. sales of luxury cars or sales of the BMW 3-series) one typically benefits from having more reliable data, but the data are less relevant. In contrast, the bottom-up approach is more relevant and less reliable. "(For a more complete discussion on these issues, see Armstrong, 1985: 250-66 and MacGregor 1998.) Research on this topic has been done under the heading of “decomposition” or “segmentation.” Additive breakdowns tend to be fairly safe. Seldom do they harm forecast accuracy, and often they provide substantial improvements (Dangerfield and Morris 1992). 3. Causal approaches to sales forecasting Instead of extrapolating sales directly, one can forecast the factors that cause sales to vary. This begins with environmental factors such as population, gross national product (GNP) and the legal system. These affect the behavior of customers, competitors, suppliers, distributors and complementors (those organizations with whom you cooperate). Their actions lead to a market forecast. Their actions also provide inputs for the market share forecast. The product of the market forecast and the market share forecast yields the sales forecast. The breakdown of the problems into the elements of Figure 2 may aid one”s thinking about the sales forecasts. It is expected to improve accuracy (versus the extrapolation of sales) only if one has good information about each of the components and if there is a good understanding about how each relates to sales. If there is high uncertainty about any of the elements, it might be more accurate to extrapolate sales directly. Figure 2. Causal approach to sales forecasting Environment Customers Market Forecast Company Marketing Mix Market Share Sales Forecast Supplier(s) Distributor(s) and Complementor(s) Competition Marketing Mix The primary advantage of the indirect approach is that it can be more directly related to decision making. Adjustments can be made in the marketing mix to see how this would affect the forecast. Also, forecasts can be prepared to assess possible changes by other decision makers such as competitors or complementors. These forecasts can allow the firm to develop contingency plans, and these effects on 6 sales can also be forecast. On the negative side, the causal approach is more expensive than sales extrapolation. Environment It is sometimes possible to obtain published forecasts of environmental factors from Tablebase, which is available on the Internet through various subscribing business research libraries. These forecasts may be adequate for many purposes. However, sometimes it is difficult to determine what methods were used to create the forecasts. In such cases, econometric models can improve the accuracy of environmental forecasts. They provide more accurate forecasts than those provided by extrapolation or by judgment when large changes are involved. Allen (1999) summarizes evidence on this. Important findings that aid econometric methods are to: (1) base the selection of causal variables upon forecasting theory and knowledge about the situation, rather than upon the statistical fit to historical data (also, tests of statistical significance play no role here); (2) use relatively simple models (e.g. do not use simultaneous equations; do not use models that cannot be specified as linear in the parameters); and (3) use variables only if the estimated relationship to sales is in the same direction as specified a priori. The last point is consistent with the principle of using causal not statistical reasoning. Consistent with this viewpoint, leading indicators, a non causal approach to forecasting that has been widely accepted for decades, does not seem to improve the accuracy of forecasts (Diebold and Rudebusch 1991). Interestingly, there exists little evidence that more accurate forecasts of the environment (e.g. population, the economy, social trends, technological change) lead to better sales forecasts. This, of course, seems preposterous. I expect that the results have been obtained for studies where the conditions were not ideal for econometric methods. For example, if things continue to change as they have in the past, there is little reason to expect an econometric model to help with the forecast. However, improved environmental forecasts are expected when large changes are likely, such as the adoption of free trade policies, reductions in tariffs, economic depressions, natural disasters, and wars. Customers One should know the size of the potential market for the given product category (e.g. how many people in region X might be able to purchase an automobile), the ability of the potential market to purchase (e.g. income per capita and the price of the product), and the needs of the potential customers. Examination of each of these factors can help in forecasting demand for the category. Company The company sets its own marketing mix so there is typically little need to forecast these actions. However, sometimes the policies are not implemented according to plan because of changes in the market, actions by competitors or by retailers, or a lack of cooperation by those in the firm. Thus, it may be useful to forecast the actions that will actually be taken (e.g. if we provide a trade discount, how will this affect the average price paid by final consumers?) Intermediaries What actions will be taken by suppliers, distributors and complementors? One useful prediction model is to assume that their future decisions will be similar to those in the past, that is, the naive model. For existing markets, this model is often difficult to improve upon. When large changes are expected, however, the naive model is not appropriate. In such cases one can use structured judgment, extrapolate from analogous situations, or use econometric models. 7 Structure typically improves the accuracy of judgment, especially if it can realistically mirror the actual situation. Role playing is one such structured technique. It is useful when the outcome depends on the interaction among different parties and especially when the interaction involves conflict. Armstrong and Hutcherson (1989) asked subjects to role play the interactions between producers and distributors. In this disguised situation, Philco was trying to convince supermarkets to sell its appliances through a scheme whereby customers received discounts based on the volume of purchases at selected supermarkets. Short (less than one hour) role plays of the situation led to correct predictions of the supermarket managers” responses for 75 per cent of the 12 groups. In contrast, only one of 37 groups was correct when groups made predictions without benefit of formal techniques. (As it turned out, the decision itself was poor, but that is another story.) Econometric models offer an alternative, although much more expensive approach to forecasting the actions by intermediaries. This approach requires a substantial amount of information. For example, Montgomery (1975) described a model to predict whether a supermarket buying committee would put a new product on its shelves. This model, which used information about advertising, suppliers” reputation, margin and retail price, provided reasonable predictions for a hold-out sample. Competitors Can we improve upon the simple, “naïve,” forecast that competitors will continue to act as they have in the past? These forecasts are difficult because of the interaction that occurs among the key actors in the market. Because competitors have conflicting interests, they are unlikely to respond truthfully to an intentions survey. A small survey of marketing experts suggested that the most popular approach to forecasting competitors” actions is unaided expert opinion (Armstrong et al. 1987). Because the ,experts” are usually those in the company, however, this may introduce biases related to their desired outcomes. For example, brand managers are generally too optimistic about their brands. Here again, role playing would appear to be relevant. Although no direct experimental evidence is available on its value in forecasting competitor”s actions, role playing has proven to be accurate in forecasting the decision made in conflict situations (Armstrong 1999). Market share Can we do better than the naive model of no change? For existing markets that are not undergoing major change, the naive model is reasonably accurate (Brodie et al. 1999). This is true even when one has excellent data about the competitors (Alsem et al. 1989). However, causal models should improve forecasts when large changes are made, such as when price reductions are advertised. Causal models should also help when a firm”s sales have been artificially limited due to production capacity, tariffs, or quotas. Furthermore, contingent forecasts are important. Firms can benefit by obtaining good forecasts of how its policies (e.g. a major price reduction) would affect its market share. 4. New product forecasting New product forecasting is of particular interest in view of its importance to decision making. In addition, large errors are typically made in such forecasts. Tull (1967) estimated the mean absolute percentage error for new product sales to be about 65 per cent. Not surprisingly then, pretest market models have gained wide acceptance among business firms; Shocker and Hall (1986) provide an evaluation of some of these models. 8 The choice of a forecasting model to estimate customer response depends on the stage of the product life-cycle. As one moves through the concept phase to the prototype, test market, introductory, growth, maturation, and declining stages, the relative value of the alternative forecasting methods changes. In general, the movement is from purely judgmental approaches to quantitative models that use judgment as inputs. For example, intentions and expert opinions are vital in the concept and prototype stages. Later, expert judgment is useful as an input to quantitative models. Extrapolation methods may be useful in the early stages if it is possible to find analogous products (Claycamp and Liddy 1969). In later stages, extrapolation methods become more useful and less expensive as one can work directly with time-series data on sales or orders. Econometric and segmentation methods become more useful after a sufficient amount of actual sales data are obtained. When the new product is in the concept phase, a heavy reliance is usually placed on intentions surveys. Intentions to purchase new products are complicated because potential customers may not be sufficiently familiar with the proposed product and because the various features of the product affect one another (e.g. price, quality, and distribution channel). This suggests the need to prepare a good description of the proposed product. This often involves expensive prototypes, visual aids, product clinics, or laboratory tests. However, brief descriptions are sometimes as accurate as elaborate descriptions as found in Armstrong and Overton”s (1970) study of a new form of urban mass transportation. In the typical intentions study, potential consumers are provided with a description of the product and the conditions of sale, and then are asked about their intentions to purchase. Eleven-point rating scales are recommended. The scale should have verbal designations such as 0 = No chance, almost no chance (1 in 100) to 10 = Certain, practically certain (99 in 100). It is best to state the question broadly about one”s “expectations” or “probabilities” to purchase, rather than the narrower question of intentions. This distinction was raised early on by Juster (1966) and its importance has been shown in empirical studies by Day et al. (1991). Intentions surveys are useful when all of the following conditions hold: (1) the event is important; (2) responses can be obtained; (3) the respondent has a plan; (4) the respondent reports correctly; (5) the respondent can fulfill the plan; and (6) events are unlikely to change the plan. These conditions imply that intentions are more useful for short-term forecasts of business-to-business sales. The technology of intentions surveys has improved greatly over the past half century. Useful methods have been developed for selecting samples, compensating for nonresponse bias, and reducing response error. Dillman (1978) provides excellent advice that can be used for designing intentions surveys. Improvements in this technology have been demonstrated by studies on voter intentions (Perry 1979). Response error is probably the most important component of total error (Sudman and Bradburn 1982). Still, the correspondence between intentions and sales is often not close. Morwitz (1999) provides a review of the evidence on intentions to purchase. As an alternative to asking potential customers about their intentions to purchase, one can ask experts to predict how consumers will respond. For example, Wotruba and Thurlow (1976) discuss how opinions from members of the sales force can be used to forecast sales. One could ask distributors or marketing executives to make sales forecasts. Expert opinions studies differ from intentions surveys. When an expert is asked to predict the behavior of a market, there is no need to claim that this is a representative expert. Quite the contrary, the expert may be exceptional. When using experts to forecast, one needs few experts, typically only between five and twenty (Hogarth 19,78; Ashton 1985). Experts are especially useful at diagnosing the current situation, which we might call “nowcasting.” Surprisingly, however, when the task involves forecasting change, experts with modest domain expertise 9 (about the item to be forecast) are just as accurate as those with high expertise (Armstrong 1985: 91-6 reviews the evidence). This means that it is not necessary to purchase expensive expert advice. Unfortunately, experts are often subject to biases. Salespeople may try to forecast on the low side if the forecasts will be used to set quotas. Marketing executives may forecast high in their belief that this will motivate the sales force. If possible, avoid experts who would have obvious reasons to be biased (Tyebjee 1987). Another strategy is to include a heterogeneous group of experts in the hopes that their differing biases may cancel one another. Little is known about the relative accuracy of expert opinions versus consumer intentions. However, Sewall (1981) found that each approach contributes useful information such that a combined forecast is more accurate than either one alone. Producers often consider several alternative designs for the new product. In such cases, potential customers can be presented with a series of perhaps twenty or so alternative offerings. For example, various features of a personal computer, such as price, weight, battery life, screen clarity and memory might vary according to rules for experimental design (the basic ideas being that each feature should vary substantially and that the variations among the features should not correlate with one another). The customer is forced to make trade-offs among various features. This is called “conjoint analysis” because the consumers consider the product features jointly. This procedure is widely used by firms (Wittink and Bergestuen 1998). An example of a successful application is the design of a new Marriott hotel chain (Wind et al. 1989). The use of conjoint analysis to forecast new product demand can be expensive because it requires large samples of potential buyers, the potential buyers may be difficult to locate, and the questionnaires are not easy to complete. Respondents must, of course, understand the concepts that they are being asked to evaluate. Although conjoint analysis rests on good theoretical foundations, little validation research exists in which its accuracy is compared with the accuracy of alternative techniques such as Delphi or judgmental forecasting procedures. Expert judgments can be used in a manner analogous to the use of consumers” intentions for conjoint analysis. That is, the experts could be asked to make predictions about situations involving alternative product design and alternative marketing plans. These predictions would then be related to the situations by regression analysis. Following the philosophy for naming conjoint analysis, this could be called exjoint analysis. It is advantageous to conjoint analysis in that few experts are needed (probably between five and twenty). In addition, it can incorporate policy variables that might be difficult for consumers to assess. Once a new product is on the market, it is possible to use extrapolation methods. Much attention has been given to the selection of the proper functional form to extrapolate early sales. The diffusion literature uses an S-shaped curve to predict new product sales. That is, growth builds up slowly at first, becomes rapid as word-of-mouth and observation of use spread, then slows again as it approaches a saturation level. A substantial literature exists on diffusion models. Despite this, the number of comparative validation studies is small and the benefits of choosing the best functional form seem to be modest (research on this is reviewed by Meade 1999). 5. Evaluating and selecting methods Assume that you were asked to predict annual sales of consumer products such as stoves, refrigerators, fans and wine for the next five years., What forecasting method would you use? As indicated above, the selection should be guided by the stage in the product life-cycle and by the 10 availability of data. But general guidelines cannot provide a complete answer. Because each situation differs, you should consider more than one method. Given that you use more than one method to forecast, how should you pick the best method? One of the most widely used approaches suggests that you select the one that has performed best in the recent past. This raises the issue of what criteria should be used to identify the best method. Statisticians have relied upon sophisticated procedures for analyzing how well models fit historical data. However, this has been of little value for the selection of forecasting methods. Forecasters should ignore measures of fit (such as RZ or the standard error of the estimate of the model) because they have little relationship to forecast accuracy. Instead, one should rely on ex ante forecasts from realistic simulations of the actual situation faced by the forecaster. By ex ante, we mean that the forecaster has only that information that would be available at the time of an actual forecast. Traditional error measures, such as mean square error, do not provide a reliable basis for comparison of methods (for empirical evidence on this, see Armstrong and Collopy 1992). The Median Absolute Percentage Error (MdAPE) is more appropriate because it is invariant to scale and is not overly influenced by outliers. For comparisons using a small set of series, it is desirable, also, to control for degree of difficulty in forecasting. One measure that does this is the Median Relative Absolute Error (MdRAE), which compares the error for a given model against errors for the naive, no change forecast (Armstrong and Collopy 1992). One can avoid the complexities of selection by simply combining forecasts. Considerable research suggests that, lacking well-structured domain knowledge, equally-weighted averages are as accurate as any other weighting scheme (Clemen 1989). This produces consistent, though modest improvements in accuracy, and it reduces the likelihood of large errors. Combining seems to be especially useful when the methods are substantially different. For example, Blattberg and Hoch (1990) obtained improved sales forecasts by equally weighting managers” judgmental forecasts and forecasts from a quantitative model. The selection and weighting of forecasting methods can be improved by using domain knowledge (about the item to be forecast) as shown in research on rule-based forecasting (Collopy and Armstrong 1992). Domain knowledge can be structured, especially with respect to trend expectations. These, along with a consideration of the features of the data (e.g. discontinuities), enable improvements in the weightings assigned to various extrapolations. 6. Estimating prediction intervals In addition to improving accuracy, forecasting is also concerned with assessing uncertainty. Although statisticians have given much attention to this problem, their efforts generally rely upon fits to historical data to infer forecast uncertainty. Here also, you should simulate the actual forecasting procedure as closely as possible, and use the distribution of the resulting ex ante forecasts to assess uncertainty. So, if you need to make two-year-ahead forecasts, save enough data to be able to have a number of two-year ahead ex ante forecasts. The prediction intervals from quantitative forecasts tend to be too narrow. Some empirical studies have shown that the percentage of actual values that fall outside the 95 per cent prediction intervals is substantially greater than 5 per cent, and sometimes greater than 50 per cent (Makridakis et al. 1987). This occurs because the estimates ignore various sources of uncertainty. For example, discontinuities might occur over the forecast horizon. In addition, forecast errors in time series are usually asymmetric, so this makes it difficult to estimate prediction intervals. The most sensible procedure is to transform the forecast and actual values to logs, then calculate the prediction intervals using logged differences. 11 Interestingly, researchers and practitioners do not follow this advice except where the original forecasting model has been formulated in logs. When the trend extrapolation is contrary to the managers” expectations, the errors are asymmetrical in logs. Evidence on the issue of asymmetrical errors is provided in Armstrong and Collopy (1998a). In such cases, one might use asymmetrical prediction intervals. Notice that this discussion takes no account of asymmetric economic loss functions. For example, the cost of a forecast that is too low by 50 units (lost sales) may differ from the cost if it is too high by 50 units (excess inventory). But this is a problem for the planner, not the forecaster. Judgmental forecasts are also too narrow. That is, experts are typically overconfident (Arkes 1999). To a large extent, this is because forecasters do not get good feedback on their predictions. When they do, such as happens for weather forecasters, they can be well calibrated. When forecasters say that there is a 60 per cent chance of rain, it rains 60 per cent of the time. This suggests that marketing forecasters should try to ensure that they receive feedback on the accuracy of their forecasts. The feedback should be relatively frequent and it should summarize accuracy in a meaningful fashion. Another procedure that helps to avoid overconfidence is for the forecaster to make a written list of all of the reasons why the forecast might be wrong. 7. Implementation There are two key implementation problems. First, how can you gain acceptance of new forecasting methods, and second, how can you gain acceptance of the forecasts, themselves? Acceptance of forecasting methods The diffusion rate for new methods is slow. Exponential smoothing, one ofthe major developments for production and inventory control forecasting, was developed in the late 1950s, yet it is only recently that the adoption rate has been substantial (Mentzer and Kahn 1995). Adoption is probably slow because there are many steps involved in the diffusion of the method. Here is the traditional procedure. Techniques are first developed. Some time later they are tested. At each stage they are reported in the literature. They are later passed along via courses, textbooks, and consultants, eventually reaching the manager who can use them. Even then they may be resisted, perhaps because the procedures are too complex for the users. The future is promising, however. The latest methods can be fully disclosed on websites and they can be incorporated into expert systems and software packages. For example, the complete set of rules for rule-based forecasting is kept available and up-to-date and can be accessed through the forecasting principles site (forecastingprinciples.com). Acceptance of forecasts Forecasts are especially useful for situations that are subject to significant changes. Often, these involve bad news. For example, Griffith and Wellman (1979), in a follow-up study on the demand for hospital beds, found that the forecasts from consultants were typically ignored when they indicated a need that was less than that desired by the hospital administrators. Firms often confuse forecasting with planning, and they may use the forecast as a tool to motivate people. That is, they use a “forecast” to drive behavior, rather than making a forecast conditional on behavior. (One wonders if they also change their thermometers in order to influence the weather.) One 12 way to avoid this problem is to gain agreement on what forecasting procedures to use prior to presenting the forecasts. Another way to gain acceptance of forecasts is to ask decision makers to decide in advance what decisions they will make, given different possible forecasts. Do the decisions differ? These prior agreements on process and on decisions can greatly enhance the value of the forecasts, but they are difficult to achieve ,in many organizations. The use of scenarios offers an aid to this process. Scenarios involve writing detailed stories of how decision makers would handle situations that involve alternative states of the future. Decision makers project themselves into the situation and they write the stories in the past tense. (More detailed instructions for writing scenarios are summarized in Gregory 1999.) Scenarios are effective in getting forecasters to accept the possibility that certain events might occur. 8. Conclusions Extrapolations of sales are inexpensive and often adequate for the decisions that need to be made. In situations where large changes are expected or where one would like to examine alternative strategies, causal approaches are recommended. Some of the more important findings about sales forecasting methods can be summarized as follows: • Methods should be selected on the basis of empirically-tested theories, not statistically based theories. • Domain knowledge should be used. • When possible, forecasting methods should use behavioral data, rather than judgments or intentions to predict behavior. • When using judgment, a heavy reliance should be placed on structured procedures such as Delphi, role playing, and conjoint analysis. • Overconfidence occurs with quantitative and judgmental methods. In addition to ensuring good feedback, forecasters should explicitly list all the things that might be wrong about their forecast. • When making forecasts in highly uncertain situations, be conservative. For example, the trend should be dampened over the forecast horizon. • Complex models have not proven to be more accurate than relatively simple models. Given their added cost and the reduced understanding among users, highly complex procedures cannot be justified at the present time. The sales forecast should be free of political considerations in a firm. To help ensure this, emphasis should be on agreeing about the forecasting methods, rather than the forecasts. Also, for important forecasts, decisions on their use should be made before the forecasts are provided. Scenarios are helpful in guiding this process. Further reading (References cited in the text marked *) * Allen, G. P. (2001) ‘Econometric forecasting strategies and techniques,” in J. S. Armstrong (ed.) Principles of Forecasting: Handbook for Researchers and Practitioners, Norwell, MA: Kluwer Academic Publishers. 13 * Alsem, K. J., Leeflang, P. S. H. and Reuyl, J. C. (1989) “The forecasting accuracy of market share models using predicted values of competitive marketing behavior,” International Journal of Research in Marketing 6: 183-98. * Arkes, H. (1999) “Overconfidence in judgmental forecasting,” in J. S. Armstrong (ed.) Principles of Forecasting: Handbook for Researchers and Practitioners, Norwell, MA: Kluwer Academic Publishers. * Armstrong, J. S. (1985) Long-Range Forecasting: From Crystal Ball to Computer, New York: John Wiley. Armstrong, J. S. (2001) “Role playing: A method to forecast decisions,” in J. S. Armstrong (ed.) Principles of Forecasting: Handbook for Researchers and Practitioners, Norwell, MA: Kluwer Academic Publishers. * Armstrong, J. S. (2001) ‘Judgmental bootstrapping: Inferring experts’ rules for forecasting,” in J. S. Armstrong (ed.) Principles of Forecasting: Handbook for Researchers and Practitioners, Norwell, MA: Kluwer Academic Publishers. * Armstrong, J. S. and Collopy, F. (1998a) “Prediction intervals for extrapolation of annual economic data: Evidence on asymmetry corrections,” Working paper. * Armstrong, J. S. and Collopy, F. (1998b) “Integration of statistical methods and judgment for time series forecasting: principles from empirical re search,” in G. Wright and P. Goodwin (eds.) Forecasting with Judgement. Chichester: John Wiley. * Armstrong, J. S. and Collopy, F. (1993) “Causal forces: Structuring knowledge for time series extrapolation,” Journal of Forecasting, 12: 103-15. * Armstrong, J. S. and Collopy, F. (1992) “Error measures for generalizing about forecasting methods: Empirical comparisons,” International Journal of Forecasting, 8: 69-80. * Armstrong, J. S. and Hutcherson, P. (1989) ‘Predicting the outcome of marketing negotiations: Roleplaying versus unaided opinions,” International Journal of Research in Marketing, 6: 227-39. * Armstrong, J. S. and T. Overton (1970) “Brief vs. comprehensive descriptions in measuring intentions to purchase,” Journal of Marketing Research 8: 114-17. * Armstrong, J. S., Brodie, R. and McIntyre, S. (1987) “Forecasting methods for marketing,” International Journal of Forecasting 3: 355-76. * Ashton, A. H. (1985) “Aggregating subjective forecasts: Some empirical results,” Management Science 31: 1499-508. * Blattberg, R. C. and Hoch, S. J. (1990) “Database * models and managerial intuition: 50 per cent model + 50 per cent manager,” Management Science 36: 887-99. * Brodie, R. J., Danaher, P., Kumar, V. and Leeflang, P. (2001) “Market share forecasting,” in J. S. Armstrong (ed.) Principles of Forecasting: Handbook for Researchers and Practitioners, Norwell, MA: Kluwer Academic Publishers. 14 * Claycamp, H. J. and Liddy, L. E. (1969) “Prediction of new-product performance: An analytical approach,” Journal of Marketing Research 6: 414-20. * Clemen, R. T. (1989) “Combining forecasts: A review and annotated bibliography,” International Journal of Forecasting 5: 559-83. * Collopy, F. and Armstrong, J. S. (1992) “Rule-based forecasting: Development and validation of an expert systems approach to combining time-series extrapolations,” Management Science 39: 1394- 414. * Dalrymple, D. J. (1975) “Sales forecasting: Methods and accuracy,” Business Horizons 18: 69-73. * Dalrymple, D. J. (1987) “Sales forecasting practices: Results from a U.S. survey,” International Journal of Forecasting 3: 379-91. * Dangerfield, B. J. and Morris, J. S. (1992), “Top-down or bottom-up: Aggregate versus disaggregate extrapolations,” International Journal of Forecasting 8: 233-41. Day, D. B., Gan, B, Gendall, P. and Esslemont, D. (1991) “Predicting purchase behavior,” Marketing Bulletin 2: 18-30. Full text available at http://marketing-bulletin.massey.ac.nz/bulletin/index.html. * Diebold, F. X. and Rudebusch, G. D. (1991) “Forecasting output with the composite leading index: A real time analysis,” Journal of the American Statistical Association 86: 603-10. * Dillman, D. A. (1978) Mail and Telephone Surveys, New York: John Wiley. * Gardner, E. S. Jr. (1985) “Exponential smoothing: The state of the art,” Journal of Forecasting 4: 1-28. * Gardner, E.S. and Anderson, E.A. (1997) “Focus forecasting reconsidered,” International Journal of Forecasting, 13: 501-08. Gardner, E. S. Jr. and McKenzie, E. (1985) “Forecasting trends in time series,” Management Science 31: 1237-46. * Gregory, L. (1999) “Scenarios as a strategy for the Researchers and Practitioners, Norwell, MA: Kluwer Academic Publishers. Griffith, J. R. and Wellman, B. T. (1979) “Forecasting bed needs and recommending facilities plans for community hospitals: A review of past performance,” Medical Care 17: 293-303. Hogarth, R. M. (1978) “A note on aggregating opinions,” Organizational Behavior and Human Performance 21: 40~. Juster, F. T. (1966) “Consumer buying intentions and purchase probability: An experiment in survey design,” Journal of the American Statistical Association, 61: 658-96. MacGregor, D. (1998) “Decomposition for judgmental forecasting and estimation,” in J. S. Armstrong (ed.) Principles of Forecasting: Handbook for Researchers and Practitioner, Norwell, MA: Kluwer Academic Publishers. 15 Makridakis, S. and Wheelwright, S.C. (1989) Forecasting Methods for Management, New York: John Wiley. Makridakis, S., et al. (1984) The Forecasting Accuracy of Major Time Series Methods, New York: John Wiley. Makridakis, S., Hibon, M., Lusk, E. and Belhadjali, M. (1987) “Confidence intervals: An empirical investigation of the series in the M-competition,” International Journal of Forecasting 3: 489-508. Meade, N. (2001) “Forecasting the diffusion of innovations: Implications for time series extrapolation,” in J. S. Armstrong (ed.) Principles of Forecasting: Handbook for Researchers and Practitioners, Norwell; MA: Kluwer Academic Publishers. Mentzer, J.T. and Kahn, K.B. (1995) “Forecasting technique familiarity, satisfaction, usage, and application,” Journal of Forecasting, 14:465-76. Montgomery; D. B. (1975) “New product distribution: An analysis of supermarket buyer decisions,” Journal of Marketing Research, 12: 255-64. Morwitz, V. (2001) “Methods for forecasting from intentions and probability data,” in J. S. Armstrong (ed.) Principles of Forecasting: Handbook for Researchers and Practitioners. Norwell, MA: Kluwer Academic Publishers. * Perry, P. (1979) “Certain problems in election survey methodology,” Public Opinion Quarterly 43: 312-25. * Rowe, G. and Wright, G. (2001) “The Delphi technique as a forecasting tool,” in J. S. Armstrong (ed.) Principles of Forecasting: Handbook .for Researchers and Practitioners, Norwell, MA: Kluwer Academic Publishers. Sanders, N. R. and Ritzman, L. P. (1992) “The need for contextual and technical knowledge in judgmental forecasting,” Journal of Behavioral Decision Making 5:39-52. * Schnaars, S. P. (1984) ‘Situational factors affecting forecast accuracy,” Journal of Marketing Research, 21: 290-7. * Sewall, M. A. (1981) “Relative information contributions of consumer purchase intentions and management judgment as explanators of sales,” Journal of Marketing Research 18: 249-53. * Shocker, A. D. and Hall, W. G. (1986) “Pretest market models: A critical evaluation,” Journal of Product Innovation Management 3: 86-107. * Sudman, S. and Bradbum, N. R. (1982) Asking Questions: A Practical Guide to Questionnaire Design, San Francisco: Jossey-Bass. * Tull, D. S. (1967) “The relationship of actual and predicted sales and profits in new-product introductions,” Journal of Business 40: 233-50. * Tyebjee, T. T. (1987) “Behavioral biases in new product forecasting,” International Journal of Forecasting 3: 393-404. 16