Price determination is essential as it plays a critical role in improving a company’s profitability and market competitiveness. Companies need competitive pricing strategies and policies that take into consideration supply and demand, costs and profits, market share and position to gain a strong market advantage. Therefore, industries like airlines, hotels, the performing arts and retail have developed price optimisation models as part of their successful pricing decision support systems. Price optimisation is a powerful tool for helping companies maximise profits as it can recommend the optimal price using mathematical formulations. Since the price optimisation model considers all possible price choices, companies can make the right decisions under varying circumstances. Information on a range of factors is needed for an accurate price optimisation model, but identifying consumers’ willingness to pay for different price options is the most important challenge. In other words, the first essential element of a successful price optimisation model is to make accurate demand predictions.
Recently, the retail industry has been actively using price optimisation models to determine prices, inventory levels and assortment. However, in the fast-moving consumer goods (FMCG) sector, this requires the ability to calculate customer demand taking into account the substitution effect that exists between multiple products within a product category. The substitution effect is a change in demand for goods due to changes in the price of other products, and it leads to an exponential number of combinations of variables. Therefore, research on price optimisation for multiple products with substitution effects has received less attention than for a single product.
Only a limited number of studies have attempted to develop a demand prediction model for multiple products with substitution effects. Moreover, most assume that customers prefer certain products based on consumer utility functions and individual choice models. They have used a mathematical model that assumes sellers are aware of individual consumers’ preferences, but individual customer preferences are not easy to define and calculate accurately in real business situations.
Recently, many researchers have constructed demand prediction models for the FMCG industry with data-driven methods like machine learning. However, most have yet to consider the substitution effect between multiple products in the category. Although there have been attempts to develop demand prediction models by applying a substitution effect structure, their research has rarely followed through by constructing a price optimisation model.
Taking into account this gap in the research, this study aims to provide a technique to allow the retailer to solve price optimisation by applying the substitution effect to the demand prediction and price optimisation models. The research question, as derived from the initial literature review, is to correctly calculate the substitution effects between products in the category to gain insight from them. In addition, it is also an important goal of this research to identify the key variables that affect product sales. Through an exploratory data analysis (EDA) process using various statistical and time series analyses, we calculated and detected price sensitivity, cross-product price elasticity, and other key variables that affect product demand and sales.
The main research question of this study is how to reflect the substitution effect structure in demand prediction and price optimisation models. It was necessary to forecast demand according to the combination of price changes of all products to reflect the effect of substitution. However, the cross-product price elasticity calculated in the EDA process cannot simultaneously show the substitution effect from the price changes of all products in the category because it only shows the substitution effect between two products. Therefore, this study attempted to develop a demand prediction model that simultaneously reflects all product price options and key variables selected from EDA in a data-driven manner using historical sales transaction data.
Evaluation metrics and benchmarking models, such as the Simple Averages, Simple Median, Naïve and Seasonal Naïve models, were used to evaluate demand prediction performance. In addition, time series, multivariate linear regression, and the latest machine learning models were also tested and evaluated. The results show that machine learning ensemble models can effectively calculate the substitution effect between products and have the best predictive performance. This was followed by using an algorithm based on the selected machine learning models, hyperparameter tuning and rolling forecasting methods to select the best demand prediction model for each individual product.
The demand forecast results, as predicted from the best demand prediction model, were then used as input for the price optimisation model. Since we have already successfully reflected the substitution effect in the demand prediction model, the proposed price optimisation model can now reflect the price combination of all products in the category. The price optimisation model developed by applying the substitution effect structure and modelling the business rules of the FMCG industry as constraints was successful in obtaining a single optimal price combination that maximises weekly profits.
The last remaining research question is how to make the demand prediction and price optimisation models operate within reasonable execution times given the system resources in the FMCG industry. Implementing an efficient price optimisation algorithm was important because applying the substitution effect requires generating a massive number of price combinations.
This study aimed to drastically reduce the total number of price combinations and thus execution time through constraint programming (CP). Through computing simulations, we tested how the total number of price combinations and the execution time for demand forecasting and price optimisation changed by adjusting the number of products, discount rate options and constraints. Furthermore, we developed a new algorithm by changing the price optimisation process used in our first model. The proposed price optimisation algorithm eliminates unnecessary price combinations using the constraints prior to demand prediction. The results from testing using computer simulations show that the proposed algorithm reduces the number of price combinations and hence execution time by more than 10,000 fold as compared to the brute force approach.
The main contribution of this research is the development of an implementable decision framework through a price optimisation algorithm that allows a retailer in the FMCG industry to determine the prices more accurately for products at different price ranges with a substitution effect. It will also enable the retailer to calculate how demand varies at different price levels and how to control its inventory level. The implementation of the decision framework as developed will not only provide direct financial benefits to the company but also enable better understanding and insights into customer behaviour. Some of this research has been published in [Reference Lee, Abdollahian, Schreider and Taheri1].
