By Erik de Vos
Why publish a blog on statistical forecasting for demand planning when artificial intelligence and machine learning seem to be popping up everywhere as the ultimate holy grail? Well, because to apply machine learning in a way you can benefit from it, companies need the experience of having worked with a best-in-class statistical forecasting approach as a solid steppingstone toward the next level of demand planning. And many companies haven’t reached that level yet.
To begin our discovery of the value of statistical forecasting in 2024, let’s start with EyeOn’s concept of Smart-touch Planning & Forecasting. We believe that the best forecasting results are achieved by working in an environment that balances human and machine effort. At the core of smart-touch forecasting is the belief that a machine will always be better at recognizing patterns in data, while humans will always be needed to bring intelligence that cannot be found in the available data. This assumption then leads to 3 principles: use the machine to automate the obvious, use the machine to recommend the probable, so the human can decide, and use the machine to flag where intervention is needed.
Learn more about EyeOn Smart-Touch Planning & Forecasting.
From manual, full-touch forecasting to advanced setups with machine learning
When we observe how demand forecasting is typically done, we tend to see 3 types of setups. The first one is the full-touch forecasting setup. In this setup, the demand forecast is the result of a lot of manual translation of demand information into what is hopefully a forecast. The usage of machine capability we tend to see in this approach is the use of Excel, which at least allows for quick calculations, if Excel can handle the size of the data. Of course, keeping the forecast up to date and aligned is a real challenge in this setup. On top of that, bias creeping into the forecast is inevitable, as there are too many factors to manage in too little time.
At the other end of the spectrum, we are starting to see setups that use machine learning. Here, the forecast is the result of a machine learning engine that can use more than just sales history, with human intervention only when necessary. As the machine can see the connection between multiple demand driver inputs, such as promotions, portfolio changes, price changes, weather, and economic indicators, it will provide a more complete forecast with greater accuracy and almost no bias. The human then focuses on two things: filling in where the machine lacks data and deciding how to run the business based on the forecast. Getting to a working machine learning setup, however, requires a solid data approach and business understanding that builds confidence in what the machine will deliver. Moving from manual ways of working to trusting a machine can be quite challenging, often due to a lack of understanding of what drives demand.
This lack of understanding on what drives the demand, and the consequential gaps in processes and data that will often be there, is where we see the setup with statistical forecast models that can be the bridge to more advanced forecasting. In this setup, the machine intervenes already to unlock pattern recognition based on historical data to provide a strong baseline forecast as a starting point for the planner. Because the machine intervention is focused on historical patterns, it remains understandable to the planner which means he/she can build confidence in using machines for forecasting. In the same time, by bringing more focus on value-adding enrichment, transparency in the process and data gaps is created, which forms the basis in moving to forecasting with machine learning.
Statistical forecasting as the gateway to advanced, machine learning based forecasting
Let’s pause to consider the value that a statistical setup can bring to an organization moving toward more advanced forecasting. To do this, let us first break a demand forecast into two parts:
- What you know based on history: In any forecast, there will be a part that we can relate to the historical patterns we see. In its simplest form, this means identifying patterns like seasonality, trend and level in your own sales history and deriving future demand patterns from that. In a more advanced form, it could mean using covariates in your forecast set-up like sales data from your customers.
- What you know about the future: Since the past is no guarantee for the future, in most cases demand forecasts will need to be supplemented with elements that will be different in the future. Different plans, changing economic outlooks, expected market growth, and unforeseen events; all can and should lead to at least a validation of the forecast and in most cases an enrichment of the forecast.
Harnessing the power of statistical forecast models
Now that we know the breakdown, what is the value of statistical forecast models in your forecasting setup? In summary, we would say:
- Statistical forecast models better capture historical sales patterns; therefore, they improve accuracy and reduce bias. Statistical forecast models will play a role in the part of the forecast you connect to your history. Historical sales patterns, such as baseline, trend, and seasonality, are something that any planner can probably identify by looking at the data but turning that into a good forecast is another matter. The most common statistical models make this easy, and the resulting forecast will be more accurate than any manual baseline.
- Statistical forecast models free up time to focus on value-adding enrichment of future elements by removing the obvious reasoning about historical patterns. The time that planners spend analyzing baseline sales patterns, defining a baseline forecast, and manually applying elements such as seasonality and trend can be eliminated with statistical forecast models. This creates the opportunity to move from pure number crunching based on the past, to reasoning with the business about what to add to the forecast based on the business information you have. So, statistical forecast models automate the obvious historical sales reasoning and allow planners to focus on what they can add based on information about the future.
- Statistical forecast models make transparent what really drives your demand and prepares you for more automation. Think of statistical forecasting as a way to learn what the core data is that you will need in a more advanced machine learning setup. By focusing on enrichment, the entire organization gains experience in how to structure data, how to prioritize data, and how to connect data. Ultimately, this enrichment data experience will be critical for any setup beyond statistical forecasting with enrichment.
By automating the routine, repetitive task of identifying historical sales patterns, statistical forecast models create room for demand planners to add their valuable insights and creativity to the mix. They can build stronger relationships with account managers, gain a deeper understanding of customer needs, and refine the forecast as needed. They can shift their focus from the full portfolio to the part of the portfolio that is important but where it is more difficult to forecast. The demand planner of the future should be a business thinker, collaborator, and communicator, not just a number cruncher. Statistical forecasting isn’t about replacing human judgment; it’s about enhancing it. With the right tools and technology, demand planners can make informed decisions based on solid, data-driven insights. Bringing statistical forecast models into the equation is the first step in unlocking this potential.
Who can benefit from statistical forecasting?
Now that we understand the potential of statistical forecasting, is this potential available to everyone? Let’s talk about the prerequisites for getting started with statistical forecasting and the circumstances in which using statistical forecast models can be beneficial. Theoretically, there are only two requirements to start using statistical forecasting:
- You need some form of time-series sales data. Everyone has sales data that contains valuable information about your demand – there may even be different versions of sales data – sell-in, sell-out, registration, or even customer sales data.
- You need some structured master data. Usually this involves product and customer master data.
Ideally, statistical forecasting is set up with a proper forecasting engine and planning front-end. This should include a relevant set of forecasting models, and it should support the user in working with the best-fit model. But even without a proper statistical forecasting tool, if you have the data, you can already reap the benefits of statistical forecasting. There are enough offerings on the market that can bring a statistical forecast to your doorstep based on your data, such as EyeOn Forecast Service
In addition, we believe that statistical forecast models can be a solid starting point for many companies. When you see that a significant portion of your demand has consistent demand patterns, you should consider statistical forecasting. Even when you see that these patterns are more volatile but still repetitive, statistical forecast models can help you. On top of that, even when you see that the patterns can be quite disruptive without a clear link to the past, starting with statistical forecasting can add value by providing a starting point from which to build your case for enrichment. Ultimately, a machine learning setup will always outperform in this last situation, but it will require more data to work with. So, consider the intermediate step of statistical forecasting.
Ready to kickstart your forecast automation journey through statistical forecasting?
A high-quality statistical forecasting approach will help identify past demand patterns and apply them to future forecasts more accurately and efficiently than manual methods, freeing up planners to enrich the forecast where necessary. But most importantly: it will kickstart your forecast automation journey.
In our e-book, we guide you on your journey to getting the basics of statistical forecasting right. You will discover:
- How to get started with statistical forecasting
- Practical steps on how to get the most out of your statistical setup & effectively tackle challenges
- How to turn your statistical forecast setup into AI-based forecasting
In doing so, we will provide you with the steppingstone to optimal “smart-touch” forecasting: a balance of machine and human efforts to deliver the best possible forecast in this ever-changing environment. Get your copy of the e-book here.
