Comparative Analysis of a Large Language Model and Machine Learning Method for Prediction of Hospitalization from Nurse Triage Notes: Implications for Machine Learning-based Resource Management
Comparative Analysis of a Large Language Model and Machine Learning Method for Prediction of Hospitalization from Nurse Triage Notes: Implications for Machine Learning-based Resource Management
Abstract Predicting hospitalization from nurse triage notes has significant implications in health informatics. To this end, we compared the performance of the deep-learning transformer-based model, bio-clinical-BERT, with a bag-of-words logistic regression model incorporating term frequency-inverse document frequency (BOW-LR-tf-idf). A retrospective analysis was conducted using data from 1,391,988 Emergency Department patients at the Mount Sinai Health System spanning 2017-2022. The models were trained on four hospitals’ data and externally validated on a fifth. Bio-clinical-BERT achieved higher AUCs (0.82, 0.84, and 0.85) compared to BOW-LR-tf-idf (0.81, 0.83, and 0.84) across training sets of 10,000, 100,000, and ～1,000,000 patients respectively. Notably, both models proved effective at utilizing triage notes for prediction, despite the modest performance gap. Importantly, our findings suggest that simpler machine learning models like BOW-LR-tf-idf could serve adequately in resource-limited settings. Given the potential implications for patient care and hospital resource management, further exploration of alternative models and techniques is warranted to enhance predictive performance in this critical domain.
Klang Eyal、Timsina Prem、Raut Ganesh、Santana Fabio、Tamegue Jules、Kia Arash、Zimlichman Eyal、Freeman Robert、Gorenstein Larisa、Glicksberg Benjamin S、Cheetirala Satya Narayan、Patel Dhavalkumar、Levin Matthew A.
Mount Sinai Health System||ARC Innovation Center, Sheba Medical Center, Affiliated to Tel-Aviv University||Division of Diagnostic Imaging, Sheba Medical Center, Affiliated to Tel-Aviv UniversityMount Sinai Health SystemMount Sinai Health SystemMount Sinai Health SystemMount Sinai Health SystemMount Sinai Health SystemHospital Management, Sheba Medical Center, Affiliated to Tel-Aviv University||ARC Innovation Center, Sheba Medical Center, Affiliated to Tel-Aviv UniversityMount Sinai Health SystemHospital Management, Sheba Medical Center, Affiliated to Tel-Aviv UniversityMount Sinai Health SystemMount Sinai Health SystemMount Sinai Health SystemMount Sinai Health System
医学研究方法计算技术、计算机技术医药卫生理论
Bio-clinical-BERTTerm frequency-inverse document frequency (TF-IDF)Health informaticsPatient careHospital resource management
Klang Eyal,Timsina Prem,Raut Ganesh,Santana Fabio,Tamegue Jules,Kia Arash,Zimlichman Eyal,Freeman Robert,Gorenstein Larisa,Glicksberg Benjamin S,Cheetirala Satya Narayan,Patel Dhavalkumar,Levin Matthew A..Comparative Analysis of a Large Language Model and Machine Learning Method for Prediction of Hospitalization from Nurse Triage Notes: Implications for Machine Learning-based Resource Management[EB/OL].(2025-03-28)[2025-05-02].https://www.medrxiv.org/content/10.1101/2023.08.07.23293699.点此复制
评论