Ⅲ Call Me Maybe: Methods and Practical Implementation of Artificial intelligence in Call
6.3 Limitations and Future Research
Our research is subject to limitations which stimulate further research. First, the set of useful variables for prediction was limited. With respect to extant literature (see e.g., Bucklin and Sismeiro (2003), Moe and Fader (2004a), or van den Poel and Buckinx (2005)), we expect e.g. demographic variables, historical purchase behavior, or the time customers spend on the single pages to be informative vari-ables. Further, we did not have information about the customers’ identity and thus, could not deter-mine whether there were recurring customers. However, this information could be of great interest for analyzing online behavior and predicting shopping cart abandonment. For instance, Huang et al.
(2018) anticipated that some customers might use the mobile phone for initial purchase stages (i.e., browsing and collecting information) and then switch to the computer for completing the purchase.
However, such customers are listed as two distinct sessions in the current data. Another missing in-formation concerns the value of abandoned shopping carts. While there is a variable that indicates the value of ordered carts (i.e., VALUE_BB), the value of abandoned carts can only be estimated. In line with extant literature on shopping cart abandonment (e.g., Close and Kukar-Kinney (2010); Ku-kar-Kinney and Close (2010)), it can be assumed that the value of ordered items influences abandon-ing rates and, thus, could aid the prediction of such. Moreover, if detailed information about spent time and further, the chronological order of customers’ actions in the online shop would be available, we could decompose the session into sequences or segments. Then, we could determine a critical point in the customer’s session in which abandonment can be predicted reliably with the F1-Score or the AUC exceeding a defined threshold (see e.g., Sismeiro and Bucklin (2004)). Hence, future re-search could replicate the present study with more detailed data, e.g. between-site clickstream data (i.e., panel data collected by media measurement company), that are typically more comprehensive and frequently used in clickstream analyses (see e.g., Moe and Fader (2004a)).
Second, we excluded just-browsing customers from our investigation. A possible direction for future research could be to conduct a multi-class classification by differentiating between purchasers, aban-donments, and just-browsing customers, similar to the cluster analysis of Moe (2003).
Third, the models’ performance strongly depends on the optimized hyperparameters which may be a time-consuming procedure for some of the models. Therefore, we considered only a limited range of possible hyperparameter values. Moreover, other values of 𝑘 in cross-validation could lead to differ-ent results.
Lastly, a real-time implementation requires a certain amount of data to be collected before the model can make a reliable decision.
By implementing these models, companies may detect shopping cart abandoners in real-time and subsequently convert some of them into purchasers by making use of targeted marketing measures such as individual chat pop-ups, coupons or special discounts. For instance, Close and Kukar-Kinney (2010) suggest human-human interactions (i.e., live chats with employees or other online shoppers) to avoid shopping cart abandonment. These could pop-up on the website if the online user is predicted to abandon by the machine learning model. Therefore, future research is recommended to test whether
pop-up messages and offers impact customers’ online shopping behavior and can prevent online shopping cart abandonment.
7 Conclusion
Online shopping cart abandonment can inhibit corporate growth and hence, harm a company’s suc-cess within its competitive environment. Simultaneously, the emergence of the Internet’s commercial usage leads to the ability to track consumers’ online activities and online behavior resulting in click-stream data.
Thus, to identify online shopping cart abandoners by extracting valuable knowledge from such click-stream data we proposed different machine learning approaches. We analyzed data of a German online retailer comprising 821,048 observations and fitted the models using 10-fold cross validation.
Thereby, our paper contributes to extant literature by combining research fields of both online shop-ping cart abandonment and clickstream data with machine learning approaches.
Our data indicate that among customers abandoning their shopping carts there is a higher proportion of new customers and mobile shoppers compared to purchasers whereas the latter add more items to their shopping cart and have a higher number of page viewings on average. Moreover, our comparison results prove that gradient boosting with regularization is a suitable method to distinguish between abandonments and purchasers yielding an AUC of 0.8182, an F1-Score of 0.8569, and an accuracy of 82.29%. Nevertheless, a decision tree or boosted logistic regression may be suitable alternatives yielding only slightly less accurate prediction results and being computationally more feasible.
Nevertheless, research on clickstream data combined with machine learning approaches is still in its infancy – particularly in a marketing context. Thereby, machine learning will be inevitable for e-commerce businesses to be successful in the long-term and the analysis provided in this paper shall stimulate further research on this topic.
Appendix: Confusion Matrices
Model Prediction Actual
0 (Purchaser) 1 (Abandonment)
Logistic Regression 0 (Purchaser) 83,817 41,722
1 (Abandonment) 15,335 130,076
AdaBoost 0 (Purchaser) 62,009 21,005
1 (Abandonment) 37,143 150,793
LogitBoost 0 (Purchaser) 72,036 34,692
1 (Abandonment) 27,116 137,106
DT 0 (Purchaser) 86,464 55,634
1 (Abandonment) 12,688 116,164
GBReg
0 (Purchaser) 79,385 28,209
1 (Abandonment) 19,767 143,589
GBTree 0 (Purchaser) 77,662 27,875
1 (Abandonment) 21,490 143,923
GBDropout 0 (Purchaser) 78,294 28,352
1 (Abandonment) 20,858 143,446
KNN 0 (Purchaser) 75,687 29,383
1 (Abandonment) 23,465 142,415
MLPDropout 0 (Purchaser) 73,803 27,869
1 (Abandonment) 25,349 143,929
NB 0 (Purchaser) 475 68
1 (Abandonment) 98,677 171,730
RF 0 (Purchaser) 77,903 28,197
1 (Abandonment) 21,249 143,601
SGB 0 (Purchaser) 74,409 29,217
1 (Abandonment) 24,743 142,581
SVMRadial
0 (Purchaser) 72,724 24,427
1 (Abandonment) 26,428 147,371
References
Abu-Nimeh S, Nappa D, Wang X, et al. (2007) A comparison of machine learning techniques for phishing detection. In: Proceedings of the anti-phishing working groups 2nd annual eCrime re-searchers summit: New York, NY: ACM.
Bengio Y (2009) Learning deep architectures for AI. Boston: Now Publishers Inc.
Bogina V, Mokryn O and Kuflik T (2019) Will this session end with a purchase? Inferring current purchase intent of anonymous visitors. Electronic Commerce Research and Applications 34:
100836.
Boroujerdi EG, Mehri S, Sadeghi Garmaroudi S, et al. (2014) A study on prediction of user’s ten-dency toward purchases in websites based on behavior models. In: 2014 6th Conference on Infor-mation and Knowledge Technology (IKT): 27 - 29 May 2014, Shahrood. Piscataway, NJ: IEEE.
Boser BE, Guyon IM and Vapnik VN (1992) A training algorithm for optimal margin classifiers.
Proceedings of the fifth annual workshop on Computational learning theory: 144–152.
Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition 30(7): 1145–1159.
Breiman L (1996) Bagging predictors. Machine Learning 24(2): 123–140.
Breiman L (2001) Random Forests. Machine Learning 45(1): 5–32.
Breiman L, Friedman J, Olshen R, et al. (1984) Classification and regression trees. New York: Chap-man & Hall.
Bucklin RE and Sismeiro C (2003) A Model of Web Site Browsing Behavior Estimated on Click-stream Data. Journal of Marketing Research 40(3): 249–267.
Caruana R and Niculescu-Mizil A (2006) An empirical comparison of supervised learning algo-rithms. In: Cohen W (ed.) Proceedings of the 23rd international conference on Machine learning:
New York, NY: ACM.
Chatterjee P, Hoffman DL and Novak TP (2003) Modeling the Clickstream: Implications for Web-Based Advertising Efforts. Marketing Science 22(4): 520–541.
Chen H, Chiang RHL and Story VC (2012) Business Intelligence and Analytics: From Big Data to Big Impact. MIS quarterly 36(4): 1165–1188.
Cheung K-W, Kwok JT, Law MH, et al. (2003) Mining customer product ratings for personalized marketing. Decision Support Systems 35(2): 231–243.
Chipman HA, George EI and McCulloch RE (1998) Bayesian CART Model Search. Journal of the American Statistical Association 93(443): 935–948.
Cho C-H, Kang J and Cheon HJ (2006) Online Shopping Hesitation. CyberPsychology & Behavior 9(3): 261–274.
Chollet F and Allaire JJ (2018) Deep Learning with R. Shelter Island: Manning Publications.
Climent F, Momparler A and Carmona P (2019) Anticipating bank distress in the Eurozone: An Ex-treme Gradient Boosting approach. Journal of Business Research 101: 885–896.
Close AG and Kukar-Kinney M (2010) Beyond buying: Motivations behind consumers’ online shop-ping cart use. Journal of Business Research 63(9-10): 986–992.
Cortes C and Vapnik V (1995) Support-vector networks. Machine Learning 20(3): 273–297.
Cover T and Hart P (1967) Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13(1): 21–27.
Cui G, Wong ML and Lui H-K (2006) Machine Learning for Direct Marketing Response Models:
Bayesian Networks with Evolutionary Programming. Management Science 52(4): 597–612.
Davis J and Goadrich M (2006) The relationship between Precision-Recall and ROC curves. Pro-ceedings of the 23rd international conference on Machine learning: 233–240.
Deng L (2011) An Overview of Deep-Structured Learning for Information Processing. Proceedings of Asian-Pacific Signal & Information Processing Annual Summit and Conference (APSIPA-ASC): 1–14.
Dietterich TG (2002) Ensemble Learning. In: Arbib MA (ed.) The Handbook of Brain Theory and Neural Networks: Cambridge, MA: The MIT Press, pp. 110–125.
Doshi-Velez F and Kim B (2017) Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
Duda RO, Hart PE and Stork DG (1973) Pattern classification and scene analysis. New York: Wiley.
Fawcett T (2006) An introduction to ROC analysis. Pattern Recognition Letters 27(8): 861–874.
Fayyad U, Piatetsky-Shapiro G and Smyth P (1996) From Data Mining to Knowledge Discovery in Databases. AI magazine 17(3): 37–54.
Freund Y and Schapire RE (1997) A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences 55: 119–139.
Friedman J (2001) Greedy Function Approximation: A Gradient Boosting Machine. The Annals of Statistics 29(5): 1189–1232.
Friedman J (2002) Stochastic Gradient Boosting. Computational statistics & data analysis 38(4):
367–378.
Friedman J, Hastie TH and Tibshirani R (2000) Additive logistic regression: a statistical view of boosting. The Annals of Statistics 28(2): 337–407.
Geisser S (1975) The Predictive Sample Reuse Method with Applications. Journal of the American Statistical Association 70(350): 320–328.
Guidotti R, Monreale A, Ruggieri S, et al. (2019) A Survey of Methods for Explaining Black Box Models. ACM Computing Surveys 51(5): 1–42.
Hand DJ (2009) Measuring classifier performance: a coherent alternative to the area under the ROC curve. Machine Learning 77(1): 103–123.
Hinton GE, Osindero S and Teh Y-W (2006) A Fast Learning Algorithm for Deep Belief Nets. Neural Computation 18(7): 1527–1554.
Hochreiter S and Schmidhuber J (1997) Long short-term memory. Neural Computation 9(8): 1735–
1780.
Huang G-H, Korfiatis N and Chang C-T (2018) Mobile shopping cart abandonment: The roles of conflicts, ambivalence, and hesitation. Journal of Business Research 85: 165–174.
Jain AK, Mao J and Mohiuddin KM (1996) Artificial Neural Networks: A Tutorial. Computer 29(3):
31–44.
James G, Witten D, Hastie T, et al. (2013) An Introduction to Statistical Learning. New York, NY:
Springer New York.
Kelly S and Noonan C (2017) The doing of datafication (and what this doing does): Practices of edification and the enactment of new forms of sociality in the Indian public health service. Journal of the Association for Information Systems 18(2): 872–899.
Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selec-tion. International Joint Conference on Artifical Intelligence (IJCAI): 1137–1145.
Kohavi R and Provost F (2001) Applications of Data Mining to Electronic Commerce. In: Applica-tions of Data Mining to Electronic Commerce: Springer, Boston, MA, pp. 5–10.
Kukar-Kinney M and Close AG (2010) The determinants of consumers’ online shopping cart aban-donment. Journal of the Academy of Marketing Science 38(2): 240–250.
L’Heureux A, Grolinger K, Elyamany HF, et al. (2017) Machine Learning With Big Data: Challenges and Approaches. IEEE Access 5: 7776–7797.
Liaw A and Wiener M (2002) Classification and Regression by randomForest. R News 2(3): 18–22.
Lycett M (2013) ‘Datafication’: making sense of (big) data in a complex world. European Journal of Information Systems 22(4): 381–386.
Moe WW (2003) Buying, Searching, or Browsing: Differentiating Between Online Shoppers Using In-Store Navigational Clickstream. Journal of Consumer Psychology 13(1-2): 29–39.
Moe WW and Fader PS (2004a) Capturing evolving visit behavior in clickstream data. Journal of Interactive Marketing 18(1): 5–19.
Moe WW and Fader PS (2004b) Dynamic Conversion Behavior at E-Commerce Sites. Management Science 50(3): 326–335.
Montgomery AL (2001) Applying Quantitative Marketing Techniques to the Internet. Interfaces 31(2): 90–108.
Montgomery AL, Li S, Srinivasan K, et al. (2004) Modeling Online Browsing and Path Analysis Using Clickstream Data. Marketing Science 23(4): 579–595.
Moro S, Cortez P and Rita P (2014) A data-driven approach to predict the success of bank telemar-keting. Decision Support Systems 62: 22–31.
Muhammad I and Yan Z (2015) Supervised machine learning approaches: A survey. ICTACT Journal on Soft Computing 05(03): 946–952.
Opitz D and Maclin R (1999) Popular Ensemble Methods: An Empirical Study. Journal of Artificial Intelligence Research 11: 169–198.
Provost F and Fawcett T (1997) Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions. Proc of the 3rd International Conference on Knowledge Discovery and Data Mining: 43–48.
Provost F and Fawcett T (2001) Robust Classification for Imprecise Environments. Machine Learn-ing 42(3): 203–231.
Rajamma RK, Paswan AK and Hossain MM (2009) Why do shoppers abandon shopping cart? Per-ceived waiting time, risk, and transaction inconvenience. Journal of Product & Brand Manage-ment 18(3): 188–197.
Razi MA and Athappilly K (2005) A comparative predictive analysis of neural networks (NNs), non-linear regression and classification and regression tree (CART) models. Expert Systems with Ap-plications 29(1): 65–74.
Rokach L (2010) Ensemble-based classifiers. Artificial Intelligence Review 33(1-2): 1–39.
Rumelhart DE, Hinton GE and Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088): 533–536.
Sakar CO, Polat SO, Katircioglu M, et al. (2019) Real-time prediction of online shoppers’ purchasing intention using multilayer perceptron and LSTM recurrent neural networks. Neural Computing and Applications 31(10): 6893–6908.
Samuel AL (1959) Some studies in machine learning using the game of checkers. IBM Journal of research and development 3(3): 210–229.
Schapire RE, Freund Y, Bartlett P, et al. (1998) Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics 26(5): 1651–1686.
Schmidhuber J (2015) Deep Learning in Neural Networks: An Overview. Neural Networks 61: 85–
117.
Sismeiro C and Bucklin RE (2004) Modeling Purchase Behavior at an E-Commerce Web Site: A Task-Completion Approach. Journal of Marketing Research 41(3): 306–323.
Srivastava N, Hinton G, Krizhevsky A, et al. (2014) Dropout: A Simple Way to Prevent Neural Net-works from Overfitting. Journal of Machine Learning Research 15: 1929–1958.
Stone M (1974) Cross-Validatory Choice and Assessment of Statistical Predictions. Journal of the Royal Statistical Society: Series B (Methodological) 36(2): 111–133.
Suykens JAK and Vandewalle J (1999) Least Squares Support Vector Machine Classifiers. Neural Processing Letters 9(3): 293–300.
Tambe P, Cappelli P and Yakubovich V (2019) Artificial Intelligence in Human Resources Manage-ment: Challenges and a Path Forward. California Management Review 61(4): 15–42.
Tibshirani RJ and Tibshirani R (2009) A Bias Correction for the Minimum Error Rate in Cross-Val-idation. The Annals of Applied Statistics 3(2): 822–829.
Vafeiadis T, Diamantaras KI, Sarigiannidis G, et al. (2015) A comparison of machine learning tech-niques for customer churn prediction. Simulation Modelling Practice and Theory 55: 1–9.
van den Poel D and Buckinx W (2005) Predicting online-purchasing behaviour. European Journal of Operational Research 166(2): 557–575.
Vapnik VN (1982) Estimation of Dependences Based on Empirical Data. New York: Springer Pub-lishing Company, Incorporated.
Williams N, Zander S and Armitage G (2006) A preliminary performance comparison of five ma-chine learning algorithms for practical IP traffic flow classification. ACM SIGCOMM Computer Communication Review 36(5): 5–16.
Zhang G, Patuwo BE and Hu MY (1998) Forecasting with artificial neural networks: The state of the art. International Journal of Forecasting 14(1): 35–62.
Zhang P (1993) Model Selection Via Multifold Cross Validation. The Annals of Statistics 21(1): 299–
313.
Zhang Q, Yang LT, Chen Z, et al. (2018) A survey on deep learning for big data. Information Fusion 42: 146–157.
Zheng B and Liu B (2018) A scalable purchase intention prediction system using extreme gradient boosting machines with browsing content entropy. In: 2018 IEEE International Conference on Consumer Electronics (ICCE): Piscataway, NJ: IEEE.