Analyzing Healthcare Attrition using Machine Learning and Traditional Statistical Techniques

Sumali   Conlon; Yue  Gaob; Haitao  Liuc

doi:10.24297/ijmit.v21i.9844

Authors

Sumali Conlon Department of Marketing, Analytics and Professional Sales, University of Mississippi, University, MS, USA
Yue Gaob School of Business, Clark University, Worcester, Massachusetts, USA
Haitao Liuc Independent Researcher, Boston, Massachusetts, USA

DOI:

https://doi.org/10.24297/ijmit.v21i.9844

Keywords:

Data Analytics, Machine Learning, Human Resource Analytics, s Job Attrition, Healthcare Professional

Abstract

It is important to understand what might cause healthcare professionals to leave their jobs. In this research, we therefore analyze data on employee attrition in the healthcare sector to determine which factors motivate these professionals to leave or stay in their current careers. We combine the flexibility of machine learning techniques with the transparency of traditional statistical techniques, such as logistic regression analysis, to understand the data. With an accuracy rate of 95.6%, based on several factors in this study, we find that one of the primary reasons these healthcare professionals leave their jobs is excessive overtime requirements. Using a deeper analysis involving logistic regression, we determine the quantitative effects of our different explanatory variables. We also find that job satisfaction does not seem to have as much explanatory power as several other variables, and that it does not seem to be a mediating variable in explaining attrition.

Downloads

Download data is not yet available.

References

Retention Report: https://info.workinstitute.com/hubfs/2022%20Retention%20Report/2022%20Retention%20Report%20-%20Work%20Institute.pdf (Retrieve on 11/16/24).

Abdullahi AK, Mosanya AU, Bello N, Musa MK. Evaluation of job satisfaction among pharmacists working in public health facilities. Exploratory Research in Clinical and Social Pharmacy. 2023; 12; doi: https://doi.org/10.1016/j.rcsop.2023.100338.

Akasheh MA, Malik EF, Hujran O, ZakiN. A decade of research on machine learning techniques for predicting employee turnover: A systematic literature review. Expert Systems with Applications. 2024; 238(E), doi: https://doi.org/10.1016/j.eswa.2023.121794.

Alao D, Adeyemo A. Analyzing employee attrition using decision tree algorithms. Computing. Information Systems & Development Informatics. 2013; 4(1):17–28.

Alduayj SS. Rajpoot K. Predicting Employee Attrition using Machine Learning. International Conference on Innovations in Information Technology (IIT). 2018: 93-98; doi: 10.1109/INNOVATIONS.2018.8605976.

Aziri B. Job Satisfaction: A Literature Review. Management Research and Practice. 2011; 3(4): 77-86. http://mrp.ase.ro/no34/f7.pdf.

Bahlman-van Ooijen W, Malfait S, Huisman-de Waal G, Hafsteinsdóttir TB. Nurses’ motivations to leave the nursing profession: a qualitative meta-aggregation. J Adv Nurs. 2023;79(12):4455–71.

Bendemra H. Building an Employee Churn Model in Python to Develop a Strategic Retention Plan. 2019; https://towardsdatascience.com/building-an-employee-churn-model-in-python-to-develop-astrategic-retention-plan-57d5bd882c2d (accessed on July 15, 2024).

Bhatnagar K, Srivastava K. Job satisfaction in health-care organizations. Industrial Psychiatry Journal. 21(1): p 75-78, Jan–Jun 2012. doi: 10.4103/0972-6748.110959.

Bolden-Barrett V. Turnover costs employers $15,000 per worker. 2017; https://www.hrdive.com/news/study-turnover-costs-employers-15000-per-worker/449142/. (accessed on August 11, 2024).

Burch JB, Tom J, Zhai Y, Criswell L, Leo E, Ogoussan K. Shiftwork impacts and adaptation among health care workers. Occupational medicine. 2019; 59(3), 159-166.

Burky A. Healthcare ranked lowest for employee satisfaction, Qualtrics survey finds. 2023; https://www.fiercehealthcare.com/health-tech/qualtrics-survey-finds-only-half-healthcare-employees-think-they-are-paid-fairly#:~:text=Healthcare%20ranked%20last%20for%20employee,Experience%20Trends%20Report%20from%20Qualtrics (Retrieve on 3/4/24).

Chaw JK, Chaw SH, Quah CH, Sahrani S, Ang MC, Zhao Y, Ting TT. A predictive analytics model using machine learning algorithms to estimate the risk of shock development among dengue patients. Healthcare Analytics. 2024; 5; https://doi.org/10.1016/j.health.2023.100290.

Chakraborty C, Bhattacharya M, Pal S, Lee S. From machine learning to deep learning: Advances of the recent data-driven paradigm shift in medicine and healthcare. Current Research in Biotechnology. 2024; 7; https://doi.org/10.1016/j.crbiot.2023.100164.

Conlon, S. Why Do Data Scientists Want to Change Jobs: Using Machine Learning Techniques to Analyze Employees' Intentions in Switching Jobs. International Journal of Management and Information Technology. 2021; 16, 59-71.

Conlon S, Simmons LL, Liu F. Predicting Tech Employee Job Satisfaction Using Machine Learning Techniques. International Journal of Management and Information Technology. 2021;16:72-88.

Cotton JL, Tuttle JM. Employee turnover: A meta-analysis and review with implications for research. Academy of Management Review. 1986; 11(1): 55-70. https://doi.org/10.2307/258331.

DataRobot. Retrieved from https://www.datarobot.com/.

STATA18. StataCorp. 2023. Stata Statistical Software: Release 18. College Station, TX: StataCorp LLC.

Dharmarathne G, Jayasinghe TN, Bogahawaththa M, Meddage DPP, Rathnayake U. A novel machine learning approach for diagnosing diabetes with a self-explainable interface. Healthcare Analytics. 2024; 5; https://doi.org/10.1016/j.health.2024.100301.

Chung D, Yun J, Lee J, Jeon Y. Predictive model of employee attrition based on stacking ensemble learning. Expert Systems with Applications. 2023; 215; 119364. https://doi.org/10.1016/j.eswa.2022.119364.

d’Ettorre G, Pellicani V. Preventing shift work disorder in shift health-care workers. Safety and health at work, 2020; 11(2); 244-247.

Fallucchi F, Coladangelo M, Giuliano R, William De Luca E. (2020). Predicting Employee Attrition Using Machine Learning Techniques. Computers. 2020; 9(4), 86. https://doi.org/10.3390/computers9040086.

Ferri P, Guadi M, Marcheselli L, Balduzzi S, Magnani D, Di Lorenzo R. The impact of shift work on the psychological and physical health of nurses in a general hospital: a comparison between rotating night shifts and day shifts. Risk management and healthcare policy, 2016; 203-211.

Galanis P, Vraka I, Fragkou D, Bilali A, Kaitelidou D. Nurses’ burnout and associated risk factors during the COVID-19 pandemic: a systematic review and meta-analysis. J Adv Nurs. 2021;77:3286–302. Doi: https://doi.org/10.1111/jan.14839

Golestani A, Masli M, Shami NS, Jones J, Menon A, Mondal J. Real-Time Prediction of Employee Engagement Using Social Media and Text Mining. 17th IEEE International Conference on Machine Learning and Applications (ICMLA). 2018; Orlando, FL, 1383-1387.

Haldorai K, Kim WG, Pillai SG, Park T, Balasubramanian K. Factors affecting hotel employees’ attrition and turnover: Application of pull-push-mooring framework. International Journal of Hospitality Management. 2019; 83: 46-55. doi: https://doi.org/10.1016/j.ijhm.2019.04.003.

Hinkin TR, Tracey JB. The cost of turnover: Putting a price on the learning curve. Cornell Hotel and Restaurant Administration Quarterly. 2000; 41:14-21. https://hdl.handle.net/1813/71741.

Holtom B, Mitchell T, Lee T, Eberly M. Turnover and retention research: A glance at the past, a closer review of the present, and a venture into the future. The Academy of Management Annals. 2008; 2(1): 231-274. doi: https://doi.org/10.1080/19416520802211552.

House House S, Wilmoth M, Stucky C. Job satisfaction among nurses and physicians in an Army hospital: A content analysis. Nursing Outlook. 2022; 70(4): 601-615, doi: https://doi.org/10.1016/j.outlook.2022.03.012.

Hulin CL, Judge TA. Job attitudes. Handbook of psychology: Industrial and organizational psychology (Borman WC, Ligen DR, Klimoski RJ (Eds.), Hoboken, NJ: Wiley; 2003; 255-276.

IHS Markit Ltd. The Complexities of Physician Supply and Demand: Projections From 2019 to 2034. Washington, DC: AAMC; 2021. Retrieved: July, 2024. https://digirepo.nlm.nih.gov/master/borndig/9918417887306676/9918417887306676.pdf

Jain D, Makkar S, Jindal L, Gupta M. Uncovering Employee Job Satisfaction Using Machine Learning: A Case Study of Om Logistics Ltd. International Conference on Innovative Computing and Communications. Advances in Intelligent Systems and Computing, 2021; Vol 1166. Springer, Singapore. https://doi.org/10.1007/978-981-15-5148-2_33.

Kaggle.com dataset. “Employee Attrition for Healthcare.” Retrieved from https://www.kaggle.com/datasets/jpmiller/employee-attrition-for-healthcare/data

Kox, J., Groenewoud, J. H., Bakker, E. J. M., Bierma-Zeinstra, S. M. A., Runhaar, J., Miedema, H. S., & Roelofs, P. (2020). Reasons why Dutch novice nurses leave nursing: A qualitative approach. Nurse Education in Practice, 47, 102848. https://doi.org/10.1016/j. nepr.2020.102848

Kupietzky J. The Role of Employee Satisfaction in the Healthcare Industry. Forbes, 2023. Retrieve 2/17/24 from https://www.forbes.com/sites/forbesbusinesscouncil/2023/06/13/the-role-of-employee-satisfaction-in-the-healthcare-industry/?sh=7b23f5a369a5

Lee ES. A Theory of Migration. Demography. 1966; 3 (1): 47-57. doi:10.2307/2060063. JSTOR 2060063. S2CID 46976641.

Locke EA. The nature and causes of job satisfaction. Handbook of industrial and organizational psychology In M.D. Dunnette (Ed.). Chicago: Rand McNally. 1976; 1:1297–1343.

Hong Lu, K. Louise Barriball, Xian Zhang, Alison E. While, Job satisfaction among hospital nurses revisited: A systematic review, International Journal of Nursing Studies, Volume 49, Issue 8, 2012, Pages 1017-1038, https://doi.org/10.1016/j.ijnurstu.2011.11.009.

Mahmoudian Y, Nemati A, Safaei AS. A forecasting approach for hospital bed capacity planning using machine learning and deep learning with application to public hospitals. Healthcare Analytics. 2023; 4; doi: https://doi.org/10.1016/j.health.2023.100245.

Nagadevara V. Early Prediction of Employee Attrition in Software Companies-Application of Data Mining Techniques. Research & Practice in Human Resource Management. 2008;16:2020–2032. doi: https://doi.org/10.1109/IADCC.2018.8692137.

Otto N. Avoidable turnover costing employers big. 2017; Retrieved from Avoidable turnover costing employers big | Employee Benefit News.

Poissonnet CM, Véron M. 2000. Health effects of work schedules in healthcare professions. Journal of Clinical Nursing. 2000; 9(1):13-23.

Punnoose R, Ajit P. Prediction of employee turnover in organizations using machine learning algorithms. International Journal of Advanced Research in Artificial Intelligence. 2016; 5(9): 22–26. doi: https://doi.org/10.14569/IJARAI.2016.050904.

Ray AN, Sanyal J. Machine Learning Based Attrition Prediction. Global Conference for Advancement in Technology (GCAT), BANGALURU, India, 2019; 1-4.

Raza S., “A machine learning model for predicting, diagnosing, and mitigating health disparities in hospital readmission. Healthcare Analytics. 2022; 2; doi: https://doi.org/10.1016/j.health.2022.100100.

Rushton CH, Batcheller J, Schroeder K, Donohue P. Burnout and Resilience Among Nurses Practicing in High-Intensity Settings. Am J Crit Care. 2015 Sep;24(5):412-20. doi: 10.4037/ajcc2015291. PMID: 26330434.

Sacco JM, Schmitt N. A dynamic multilevel model of demographic diversity and misfit effects. Journal of Applied Psychology. 2005; 9(2): 203-231. doi: https://doi.org/10.1037/0021- 9010.90.2.203.

Singh JK, Jain M. A Study of Employees’ Job Satisfaction and Its Impact of their Performance. Journal of Indian Research. 2013; Vol. 1, No. 4, 105-111.

Spector PE. Job satisfaction: Application, assessment, causes and consequences. Thousand Oaks, CA: SAGE. 1997.

Wei H, Horsley L, Cao Y, Haddad LM, Hall KC, Robinson R, Powers M, Anderson DG. The associations among nurse work engagement, job satisfaction, quality of care, and intent to leave: A national survey in the United States. International Journal of Nursing Sciences. 2023; 10(4):476-484; doi: https://doi.org/10.1016/j.ijnss.2023.09.010.

Wilson JL. The impact of shift patterns on healthcare professionals. Journal of Nursing Management. 2022; 10(4):211-219.