Many organizations utilize human judgment as an addition to the statistical forecasting process. Popular statistical methods often rely on mathematical principles that incorporate previous forecasts or sales volumes. However, statistical forecasts are typically an extrapolation of history and they do not incorporate any external information of upcoming promotions or market trends.
To improve accuracy, one could combine the stability of statistical methods with the flexibility of planners’ enrichments. However, we regularly observe situations where the planner encounters challenges in accurately assessing certain instances, which may lead to a decrease in the overall accuracy of the forecasts. This will leave the organization with too much inventory or result in a disappointed customer who cannot get the desired products.
In previous blogs, we discussed the thesis by Jochem Geurts about decisional guidance in forecasting. This thesis showed the effectiveness of guidance towards better forecasting performance in an experimental setting.
Providing planners with better support in decision-making
Within the thesis project of Loek Eggels, a master student in “Operations Management and Logistics” at the Eindhoven University of Technology, we researched the indicators leading to the added value of forecast enrichments and explored effective means of appropriately notifying the planners to provide better support in decision making.
The starting point of the thesis are numerous papers in the field of Behavioural Operations Management over the last 30 years. The literature has studied a large number of planner enrichments with mixed results to their effectiveness. Just like every other human, it has often been proven that planners are biased and do not operate like fully rational decision-makers. Recently, research has also been aimed at identifying important features that indicate the likelihood of an enrichment.
In his thesis, Loek tries to answer the following research question: “What features are of great importance to the quality of forecast enrichments?” He answers this by analysing a large dataset with planner’s enrichments. The factors that focused on are the size and direction of the enrichment itself, the planner’s previous behaviour, the product category, and the time at which such an enrichment is executed.
This thesis uses an anonymized dataset from one of our customers to test and quantify the results. Advanced machine learning (LightGBM) and model explainability (SHAP) techniques are used to identify important features. We discovered that features directly tied to the enrichment – its size, statistically forecasted quantity, and past enrichment performance – are critical. For example, an enrichment that reduces the forecast compared to the statistical forecast is more likely to be accurate. Nonetheless, the product category and planners’ biases also have a significant impact on the accuracy of an enrichment.
Practical implications of notifying planners
Based on these features, we can explore the practical implications for customers. More specifically, we investigated when to notify a planner about a potential bad enrichment. There are a few guidelines one should adhere to in effectively alerting planners. All of us get bombarded by notifications on our phones throughout the day. When you receive many of them, you do not value their importance. The same holds for alerts during forecasting tasks. If you receive too many of them, you will discard them.
Therefore, you would like to notify the planner when you are sure that the enrichment will be very inaccurate. Higher performance could be achieved based on the planners’ ability to incorporate the advice of these alerts. However, planners could at least reset the enrichment to the statistical forecast to maintain the forecast accuracy.
We can show the planner why the enrichment is predicted to be inaccurate, based on the Shapley values. The figure above shows the reasoning of the machine learning model behind a notification. It is triggered by the value of the enrichment size, the planner’s biases, and the hierarchy level at which the enrichment is executed. This explanation should give planners the option to re-evaluate this enrichment and investigate how they can improve it. Ideally, planners can transform this advice into appropriate improvements. When looking further, effective utilization of the feedback could create even better results.
These findings underline the significant potential for enhancing forecast accuracy by preventing a small number of detrimental forecasts. The results open the door to more informed decision-making, reduced inventory costs, and more accurate forecasting.