Skip to main content
Download PDF

Enhancing pneumonia prognosis in the emergency department: a novel machine learning approach using complete blood count and differential leukocyte count combined with CURB-65 score

BMC Medical Informatics and Decision Making volume 24, Article number: 118 (2024) Cite this article

Abstract

Background

Pneumonia poses a major global health challenge, necessitating accurate severity assessment tools. However, conventional scoring systems such as CURB-65 have inherent limitations. Machine learning (ML) offers a promising approach for prediction. We previously introduced the Blood Culture Prediction Index (BCPI) model, leveraging solely on complete blood count (CBC) and differential leukocyte count (DC), demonstrating its effectiveness in predicting bacteremia. Nevertheless, its potential in assessing pneumonia remains unexplored. Therefore, this study aims to compare the effectiveness of BCPI and CURB-65 in assessing pneumonia severity in an emergency department (ED) setting and develop an integrated ML model to enhance efficiency.

Methods

This retrospective study was conducted at a 3400-bed tertiary medical center in Taiwan. Data from 9,352 patients with pneumonia in the ED between 2019 and 2021 were analyzed in this study. We utilized the BCPI model, which was trained on CBC/DC data, and computed CURB-65 scores for each patient to compare their prognosis prediction capabilities. Subsequently, we developed a novel Cox regression model to predict in-hospital mortality, integrating the BCPI model and CURB-65 scores, aiming to assess whether this integration enhances predictive performance.

Results

The predictive performance of the BCPI model and CURB-65 score for the 30-day mortality rate in ED patients and the in-hospital mortality rate among admitted patients was comparable across all risk categories. However, the Cox regression model demonstrated an improved area under the ROC curve (AUC) of 0.713 than that of CURB-65 (0.668) for in-hospital mortality (p<0.001). In the lowest risk group (CURB-65=0), the Cox regression model outperformed CURB-65, with a significantly lower mortality rate (2.9% vs. 7.7%, p<0.001).

Conclusions

The BCPI model, constructed using CBC/DC data and ML techniques, performs comparably to the widely utilized CURB-65 in predicting outcomes for patients with pneumonia in the ED. Furthermore, by integrating the CURB-65 score and BCPI model into a Cox regression model, we demonstrated improved prediction capabilities, particularly for low-risk patients. Given its simple parameters and easy training process, the Cox regression model may be a more effective prediction tool for classifying patients with pneumonia in the emergency room.

Peer Review reports

Background

Pneumonia is the leading cause of mortality and hospitalization globally [1,2,3]. Nonetheless, a precise assessment of its severity is crucial for effectively managing the condition and informing critical decisions regarding diagnosis, treatment, and healthcare intervention [4,5,6]. Early identification of patients at high risk of rapid pneumonia progression can facilitate prompt intervention, thereby ensuring patient safety and optimizing clinical outcomes [7].

To objectively assess the severity of pneumonia, several studies have focused on identifying independent predictors associated with adverse health outcomes. The pneumonia severity index (PSI) and CURB-65 scores are the most commonly recognized prediction tools for classifying patients with pneumonia in accordance with international guidelines [4, 8,9,10]. The PSI comprises 20 clinical and laboratory parameters, facilitating the classification of low-risk community-acquired pneumonia (CAP) patients into five distinct risk classes [8]. While the PSI exhibits robust discriminatory capacity in classifying patients into appropriate risk groups, its complex calculation poses a challenge for its clinical application, especially in the demanding settings of the emergency department (ED) [10,11,12]. In contrast, CURB-65 was developed from five readily measurable factors, rendering it user-friendly [4]. Several validation studies have indicated that its capability to predict mortality associated with CAP is nearly comparable to that of the PSI [6, 13,14,15]. However, both tools feature several variables with dichotomous and arbitrary cutoffs, limiting their predictive accuracy [16,17,18,19].

Machine learning (ML) is a widely explored domain in medicine presently. Innovative methodologies, including recursive partitioning, decision tree analysis, and random forest, provide a more robust approach to predicting clinical outcomes than traditional predictive models [20]. Various prediction models for pneumonia have been developed by training models with diverse clinical data sources, encompassing vital signs, medical history, laboratory tests, and even chest radiographs. Certain studies have demonstrated promising findings when comparing ML models with traditional prediction models [16, 20,21,22,23]. However, these complex input data mainly originated from various sources, necessitating multiple data point acquisition processes, additional blood draws, incurring costs, and proving challenging to uniformly and promptly obtain. Therefore, we developed a blood culture prediction model in our previous study, known as the blood culture prediction index (BCPI) model, exclusively leveraging complete blood counts (CBC) and differential leukocyte count (DC) data for bacteremia detection [24]. The BCPI model has demonstrated superior accuracy in predicting bacteremia compared to methods utilizing C-reactive protein (CRP) and procalcitonin (PCT) data. However, the applicability of the BCPI model in specific infections, such as pneumonia, remains unevaluated and unconfirmed.

Therefore, this study aimed to compare the performance of the BCPI model with the widely employed CURB-65 score in assessing pneumonia severity in an ED setting and their ability to predict prognosis following admission. Furthermore, we aim to construct a novel ML-based model by incorporating the CURB-65 score into the BCPI model and evaluate whether this approach enhances efficiency. Through these methods, we sought to extend the previously published BCPI model to pneumonia to generate additional value and provide potentially better predictive tools for pneumonia assessment.

Methods

Study setting

This retrospective study was conducted at Linkou Chang Gung Memorial Hospital (CGMH)—a 3400-bed tertiary medical center in Taiwan. The data utilized in the study were sourced from the Chang Gung Research Database (CGRD), which constitutes a de-identified duplicate of the clinical database at the hospital [25]. The study was approved by the Institutional Review Board of the hospital, and informed consent was waived (IRB No.: 202201120B0C101).

Study population

In our previous study, the BCPI model was trained using data from the same database from 2014—2018 [24]. In this study, data were extracted from patients who visited the ED between January 1, 2019, and December 31, 2021, with a primary pneumonia diagnosis. The inclusion comprised the following criteria: availability of CBC/DC and blood urea nitrogen (BUN) laboratory data on the same day of the ED visit, alongside data on age, sex, and assessment of respiratory rate, blood pressure, and consciousness level (Glasgow Coma Scale), conducted during triage on the day of the ED visit. These criteria ensure that the CURB-65 score and BCPI can be calculated and used to predict the prognosis for every included patient. Overall, 9,352 patients were enrolled in the analysis (Fig. 1).

Fig. 1
figure 1

Research sample flowchart. The gray-shaded sections in the flowchart enable the calculation of the CURB-65 score for patients with pneumonia, while the yellow-shaded sections aid in computing the BCPI. # ICD: J10.0, J11.0, J12, J13, J14, J15, J16, J17, and J18

Full size image

Training data preparation

The parameters used in the BCPI model mentioned above encompassed red blood cell count, hemoglobin, hematocrit, mean corpuscular volume, mean corpuscular hemoglobin, mean corpuscular hemoglobin concentration, red cell distribution width, nucleated red blood cell, white blood cell count, segmented neutrophil, band basophil, eosinophil, lymphocyte, atypical lymphocyte, plasma cell, plasmacytoid cell, hypersegmented cell, blast cell, myelocyte, meta-myelocyte, monocyte, promonocyte, and platelet count. The DC data were expressed as percentages.

To convert BUN, confusion, respiratory rate, blood pressure, and age into binary variables for model training and validation, we employed the cutoff values defined in the CURB-65 score (BUN level > 7 mmol/L, respiratory rate ≥ 30 breaths/min, systolic or diastolic pressure of < 90 mmHg and < 60 mmHg, respectively, and age ≥ 65 years old) [4]. Based on the original definition of CURB-65, "confusion" refers to a specific mental test or new disorientation of a person, place, or time [4, 26]. However, owing to retrospective research limitations, we could not confirm from the database whether patients were assessed for "confusion" based on the specified definition above. Therefore, we employed the Glasgow Coma Scale. Subsequently, we established a cutoff value of < 15. The limitations stemming from this approach are comprehensively discussed in the limitations within the discussion section.

Developing prediction models

In this study, we assessed the effectiveness of risk stratification using the BCPI model compared to that of the CURB-65 score. Subsequently, the CURB-65 and BCPI models were integrated to create a new Cox regression model. Survival analysis was then conducted using in-hospital mortality as the endpoint. The model underwent training with data from patients hospitalized in 2019 and 2020, followed by validation using data from 2021. The significance of CRP as an important indicator in assessing pneumonia severity and prognosis has been highlighted in previous studies [27,28,29]. As a preliminary step, we explored substituting BCPI with CRP and integrating it with CURB-65 to develop a new model within our study cohort. Subsequently, the performance of this alternative model was assessed.

Performance evaluation and statistical methods

To assess the prognostic performance of CURB-65 and compare it to that of the BCPI model for patients with pneumonia in the ED, we categorized patients into low- (CURB-65 score ≤ 1), moderate- (CURB-65 score = 2), and high-risk (CURB-65 score ≥ 3) groups based on their CURB-65 scores. This classification method aligns with the approach outlined in the 2009 British Thoracic Society (BTS) guidelines for managing CAP in adults [9]. We categorized the same group of patients with pneumonia into three risk groups based on the BCPI, ensuring that each group contained the same number of patients as those defined via the CURB-65 risk scores. Subsequently, we compared the 30-day all-cause mortality, admission, and in-hospital mortality rates for each risk group as defined via both assessment tools.

Following that, the Cox regression model was trained. It was then validated using data from hospitalized patients. We then compared the predictive performance of the Cox regression model and that of CURB-65 for in-hospital mortality, using the area under the receiver operating characteristic curve (AUROC) as the accuracy metric. The coefficients, p-values, and confidence intervals of each parameter in the Cox regression analysis were also provided. Regarding feature selection, we utilized Recursive Feature Elimination for the Cox regression model to iteratively eliminate features that exhibit the least effect on the AUC for mortality prediction [30]. We calculated detailed AUC values and corresponding confidence intervals for each model within our ablation study using the DeLong test [31]. All ML models and calculations, including mortality rate calculations, AUROC, and other metrics, were performed using Python 3.7 (https://www.python.org/). Statistical analyses involving variance and correlation coefficients were conducted using SPSS (IBM SPSS Statistics 19, Chicago, IL).

Results

Patients who had CBC/DC and parameters included in the CURB-65 score collected and available during their pneumonia diagnosis in our ED were included in this study. Overall, 9,352 patients meeting this criterion were enrolled, with 6,655 hospitalized.

Patient characteristics

Table 1 presents an overview of the primary information regarding the study participants. We classified all 9,352 patients into low- (CURB-65 score ≤ 1), medium- (CURB-65 score = 2), and high-risk (CURB-65 score ≥ 3) groups based on their CURB-65 scores, with 4,654, 2,513, and 2,185 patients in each group, respectively. The mean and median ages exhibited a gradual increase with rising risk levels, with males constituting the majority, making up approximately 60% of the patients across all risk groups. Positive finding proportions for each factor within CURB-65 also increased as the risk levels rose. As risk levels escalated, hospitalization rates naturally rose as well: 63% (low-risk), 78.7% (medium-risk), and 80.0% (high-risk). Similarly, 30-day mortality rates were consistent with this pattern: 5.4% (low-risk), 13.8% (medium-risk), and 26.8% (high-risk). Subsequently, an analysis was conducted on the 6,655 hospitalized patients, who were also divided into three risk groups based on their CURB-65 scores: low- (n=2,930), medium- (n=1,977), and high-risk (n=1,748) groups. The in-hospital mortality rates were calculated for each group, which were 7.9%, 15.1%, and 26.0%, respectively. To compare and validate the performance of the BCPI model to that of the CURB-65, patients were grouped into three risk categories, with an equal number of patients in each group based on their CURB-65 scores. Furthermore, we analyzed patient characteristics and calculated 30-day mortality for all emergency patients and in-hospital mortality for admitted patients. In this context, within the BCPI group, the proportions of positive findings for each CURB-65 factor still increased with increasing risk. However, compared to the CURB-65 cohort, distinct variations emerged. For instance, among those categorized as low-risk via CURB-65, only 26.1% were > 65 years of age, whereas the BCPI low-risk group exhibited a notably higher proportion at 47.0%. This discrepancy underscores how the BCPI model categorizes patients with pneumonia into risk tiers in a manner distinct from CURB-65. Nevertheless, the predictive ability of the two methods for the 30-day mortality rate among emergency patients and the in-hospital mortality rate among admitted patients were comparable across all risk categories (Table 2).

Table 1 Patient characteristics. We categorized the BCPI of the patients into three risk groups, each with an equal number of patients based on their CURB-65 scores, and then analyzed patient characteristics
Full size table
Table 2 Admission rate and mortality analysis: CURB-65 vs. BCPI
Full size table

Cox regression model performance

In this study, a Cox regression model was constructed by integrating the BCPI model with the CURB-65 score. The training set comprised cases from the admission group between 2019 and 2020 (n=4,891), while the testing set comprised cases from 2021 (n=1,764). We evaluated the performance of the Cox model and compared it with that of the CURB-65 score. Table 3 shows the results of multivariate logistic regression of variables in the Cox regression model, including coefficients, p-values, hazard ratio, and confidence intervals. Our analysis revealed that all attributes significantly influenced the prediction, with "urea" making the greatest contribution, except for the continuous variable "BCPI."

Table 3 Multivariate logistic regression of variables in the Cox regression model for predicting in-hospital mortality
Full size table

The prediction performance of in-hospital mortality rate among admitted patients was compared between the CURB-65 score and the Cox regression model (Table 4). As we utilized the Cox regression model to stratify patients into low-, medium-, and high-risk categories, with the same number of patients as in the CURB-65 score, no significant difference was observed between the two methods in predicting the prognosis of each risk group.

Table 4 In-hospital mortality analysis for the testing set in 2021: CURB-65 vs. Cox regression model
Full size table

The AUC for predicting in-hospital mortality rate was 0.668 for CURB-65, which increased to 0.713 for the Cox regression model, generally indicating acceptable discrimination for the Cox regression model (Fig. 2) [32]. The Cox regression model exhibited statistically significant discrimination in predicting in-hospital mortality within the lowest risk group (CURB-65=0) compared to the CURB-65 score (Table 5). The CURB-65 score specifically indicated a high mortality rate of 7.7%, whereas in the Cox regression model, it was 2.9% (p<0.001).

Fig. 2
figure 2

AUC for predicting in-hospital mortality rate (CURB 65 vs. Cox regression model)

Full size image
Table 5 Comparison of in-hospital mortality among hospitalized patients in 2021: CURB-65 vs. Cox regression model
Full size table

Ablation study using recursive feature elimination for the cox regression model

Initially, Recursive Feature Elimination was performed with all features and iteratively removed the least important ones based on a ML algorithm's ranking. We observed that urea, the parameter contributing most significantly to the model after BCPI, was the last to be removed in the ablation study (Fig. 3). Furthermore, the complete Cox regression model showed a significant difference in predictive ability compared to CURB-65 (AUC 0.713 vs. 0.668, p<0.001). When only the BCPI parameter remained, the model exhibited comparable AUC values to CURB-65 (AUC 0.674 vs. 0.668, p=0.758), consistent with the results presented in Table 2. Furthermore, when the model included only UR+BCPI, the AUC surpassed 0.7, generally indicating acceptable discrimination, and demonstrated a significant difference in predictive ability compared to CURB-65 (AUC 0.707 vs. 0.668, p=0.015). Table 6 presents the detailed AUC values, confidence intervals, and p-values.

Fig. 3
figure 3

Ablation study using Recursive Feature Elimination for the Cox regression model. *Indicates statistical significance with p<0.05 compared to the AUC of CURB-65

Full size image
Table 6 AUCs, confidence Intervals, and p-Values for Recursive Feature Elimination ablation Study
Full size table

Discussion

Our study builds upon previous research findings on the BCPI model for bacteremia detection and evaluation, which relies solely on CBC/DC data. We found that this model also demonstrated promising results in predicting the likelihood of admission for ED patients with pneumonia, performing comparably to that of the widely used CURB-65 model. Furthermore, we constructed a new Cox regression model by integrating the covariates of the CURB-65 score and BCPI model. This model demonstrated superior mortality predictive capabilities than CURB-65 alone, particularly in discriminating low-risk patients. These findings suggest that the Cox regression model holds the potential as a valuable tool in emergency medicine for assessing pneumonia severity. Furthermore, BCPI may emerge as a significant component in developing future ML models for pneumonia.

Patients classified as low risk based on CURB-65 are suggested to be potential candidates for outpatient treatment [3, 4, 15, 33]. However, our data analysis revealed a relatively high admission rate of 63%, contrasting with previous recommendations. In addition, the 30-day mortality rate among the low-risk patients in our study was as high as 5.4%, which is concerning, contradicting our initial expectations. One possible reason for the high admission rate observed in our study could be attributed to Taiwan's National Health Insurance program. This program provides universal healthcare coverage to approximately 99% of the population, boosting one of the lowest administrative costs globally [34]. This results in greater accessibility to medical resources than in other countries. Additionally, we were unable to distinguish between community-acquired pneumonia (CAP) and hospital-acquired pneumonia (HAP) among patients in the ED. While HAP may represent a minority, it could still influence hospital admission and mortality rates. Furthermore, considering some patients classified as low-risk via CURB-65 may not be as safe as expected. Prior validation studies on the application of CURB-65 for CAP conducted by Aujesky et al. and Barlow et al. have revealed that a low-risk CURB-65 score of 1, conventionally deemed safe, correlates with a mortality rate of 3–4% [35, 36]. This finding is notably concerning as it indicates a substantially higher mortality rate than previously thought. Our study findings are consistent with these findings, suggesting that even among patients with uncomplicated CAP, CURB-65 may not effectively identify those at high risk of deterioration within the low-risk group, potentially resulting in underestimating the risk for patients. Additionally, previous studies based on CURB-65 consistently demonstrate that most low-risk patients presenting to the hospital are admitted, as observed in our study. This indicates a discrepancy between the recommendations derived from the CURB-65 score and the actual clinical decisions made [37, 38]. For example, Choudhury et al. found that out of 565 patients with low-risk CAP, 74.3% were admitted to the hospital [37]. Similarly, Aliberti et al. found that approximately 50% of patients who presented to the ED with a CURB-65 score of 0 or 1 were admitted based on clinical judgment rather than being treated as outpatients based on the score [38]. They identified hypoxemia and decompensated comorbidities as significant factors influencing this decision. Our study findings are consistent with these results, indicating that relying solely on CURB-65 for risk stratification in pneumonia may lead to an inaccurate assessment of the low-risk patients, potentially subjecting them to additional risk. The diagnostic and treatment guidelines for community-acquired pneumonia published by the American Thoracic Society (ATS) highlight a shortcoming of the CURB-65 score, particularly its limited ability to classify patients as low risk [10]. Thus, for patients with CAP with CURB-65 scores of 0 or 1, further assessments guided by clinical judgment or novel predictive models should be prioritized.

Compared to the CURB-65 score, our findings revealed that stratifying patients based on the BCPI did not yield a significant difference in the admission or 30-day mortality rates within each risk group. Presently, emergency physicians generally utilize the CURB-65 score as the primary basis for hospitalization decisions in clinical practice [6, 33, 35, 39]. However, the BCPI model, initially intended to predict bacteremia, exhibited comparable effectiveness for pneumonia in this context. Therefore, we posit that the efficacy of the BCPI model might surpass that of CURB-65. After integrating the CURB-65 and BCPI models, our Cox regression model demonstrated a higher AUC than that of the CURB-65 score. This novel assessment tool demonstrates effective discriminative capability, achieving an AUC of ≥ 0.7—a threshold generally recognized as indicative of "acceptable discrimination." In contrast, the AUC for the CURB-65 model alone, typically ranging from 0.5–0.7, is considered to reflect “poor discrimination” [32]. Additionally, the comparative analysis revealed a statistically significant improvement in predictive performance compared to the standalone CURB-65 model, with AUC values of 0.713 vs. 0.668, respectively (p<0.001) (Table 6). Upon validating the model using patient data from 2021, we discovered that the in-hospital mortality rate among inpatients with a CURB-65 score of 0 reached 7.7%. This highlights the challenges encountered by CURB-65 in accurately distinguishing low-risk patients, as discussed earlier. In contrast, the Cox regression model demonstrated an in-hospital mortality rate of only 2.4% in this particular group of patients, and the difference was statistically significant. While the remaining risk groups did not attain statistical significance, potentially owing to the limited sample size, the highest-risk group exhibited a clear trend toward enhanced discrimination. Therefore, the Cox regression model may serve as a more beneficial tool for identifying patients with low-risk pneumonia in future clinical practice. This can facilitate the safe discharge of patients who do not require hospitalization, thereby conserving medical resources. Additionally, it can aid in identifying patients at risk while ensuring they receive appropriate treatment.

Table 3 shows that all variables in the Cox regression model achieved statistical significance. Among them, "urea" emerged as the greatest contributor to the prediction, excluding the continuous variable "BCPI." In our previous BCPI model, we identified platelet count, monocyte percentage, lymphocyte percentage, segmented neutrophil percentage, and leukocyte count as the five significant features based on importance [24]. The identified factors align with those of other studies that highlight the significance of the neutrophil-to-lymphocyte ratio (NLR) in peripheral blood as a prognostic biomarker in infectious diseases, including pneumonia. NLR can also serve as a bacteremia predictor [40,41,42]. Additionally, the ablation study using Recursive Feature Elimination for the Cox regression model, as depicted in Fig. 3, revealed that urea was the last parameter eliminated, highlighting its predictive importance within the BCPI model. The comprehensive Cox regression model demonstrated improved predictive capability than that of CURB-65 alone (AUC=0.713 vs. 0.668, p<0.001), indicating the benefit of our approach. Furthermore, a simplified model comprising only urea, respiratory rate, and BCPI achieved an AUC exceeding 0.7, typically indicative of "acceptable discrimination," and notably outperformed CURB-65. These findings confirm the synergistic potential of integrating basic laboratory data with respiratory rate assessments and underscore the substantial enhancement in predictive accuracy achievable beyond the CURB-65 model. In previous studies on ML models for pneumonia, researchers have discussed the feature importance of clinical variables. However, these studies have lacked ablation studies, which would provide valuable insights into model complementarity and feature selection for future ML advancements in pneumonia [43, 44].

Furthermore, several studies have employed inflammatory markers, including CRP and PCT, to monitor patient response during infectious disease treatment or predict prognosis [45,46,47,48,49,50]. In pneumonia cases, CRP and PCT are often used alongside prediction tools such as the CURB-65 score to assess severity and inform antibiotic therapy decisions [27,28,29]. In our study cohort, CRP data were available for 6,086 patients (65.07%). However, when we substituted BCPI with CRP and integrated it with CURB-65 to create a new model, this modification did not enhance predictive performance (Figure S1). Furthermore, CRP alone exhibited relatively poor performance, which contradicts findings in previous studies supporting its utilization in predicting CAP severity [49, 51]. This suggests that training the model using CBC/DC data can achieve comparable performance to models incorporating inflammatory markers. Moreover, CRP and PCT require additional blood draws and testing costs. Conversely, CBC/DC is the most commonly performed laboratory test for patients with unidentified infections. They are cost-effective, requiring only a single blood draw, and have a short turnaround time of approximately 22 min in the laboratory with total automation [52]. Additionally, the BCPI utilized in constructing the Cox regression model is a continuous parameter. It can be used to categorize patients into different risk groups as needed, unlike the binary parameters of CURB-65. The continuous nature of the model enables arbitrary cutoffs to be applied based on clinical needs, potentially making it applicable to different regions and hospitals at varying levels. The parameters used to train the model are feasible and readily available in most EDs. Upon receiving laboratory data, we can derive risk assessment values from the Cox regression model. These values can then be employed to inform treatment decisions and the healthcare management, particularly those at low risk. This is crucial for emergency physicians. Additionally, BCPI holds potential for expansion into other fields, such as COVID-19 pneumonia or various infectious diseases. It can also serve as an adaptable component for future ML models.

Prominent medical practice guidelines, including those provided by respected organizations such as the British Thoracic Society (BTS) and National Institute for Health and Care Excellence (NICE), recommend using the CURB-65 score along with clinical judgment to inform treatment decisions for patients with CAP [9, 53]. Our study revealed that the Cox regression model outperformed CURB-65 and demonstrated a better ability to differentiate risk levels among patients, consequently ensuring safe and appropriate medical care. Moreover, the Cox regression model only relies on five easily obtainable parameters from CURB-65, alongside CBC/DC data, routinely tested for almost all ED patients. This significantly enhances the clinical usability of the Cox regression model.

In this study, we successfully developed a highly effective prediction model for pneumonia; however, we acknowledge that this study had some crucial limitations. First, the retrospective design of our study at a single center may introduce bias and confounding factors that might influence our results. Additionally, the generalizability of our findings to other healthcare settings could be affected by differences in patient populations across various centers. Second, owing to the retrospective research constraints, we could not confirm the presence of "confusion" in patients using the specific definition outlined in the CURB-65 criteria. Conversely, we used the GCS as an alternative for analysis. Although GCS may not perfectly align with the original concept of "confusion" in the CURB-65 criteria, the GCS variable continued to significantly contribute to the Cox regression analysis model. Furthermore, compared to clinical evaluations performed by emergency physicians, the GCS may provide a simple and more objective measure. This may better suit the demands of clinical practice, especially in busy emergency department settings. Moreover, although we endeavored to exclude cases of HAP or ventilator-associated pneumonia (VAP), it is important to acknowledge that the emergency department may receive referrals of pneumonia patients from other hospitals or nursing homes. These cases could potentially diverge from typical CAP cases. While we anticipate the proportion of such cases to be minimal, their inclusion in our study could still influence the results and conclusions drawn from our research on CAP. Finally, it is important to highlight that we only included patients with pneumonia who underwent a BUN test for CURB-65 calculation. However, the frequency of BUN testing is not as high as that of CBC/DC tests (over 99%). Consequently, excluding patients who did not undergo this test may introduce potential bias into our analysis. Therefore, more studies may be necessary to validate the performance of this ML approach further. Future investigations should prioritize initiating pilot trials to assess the effectiveness of the proposed analytical methods in enhancing clinical prognostic decision-making. Essential to this effort will be establishing partnerships with medical professionals to ensure the models are appropriately integrated into clinical workflows. These studies will seek to validate the effectiveness of these approaches in real-world settings and enrich the discussion on integrating algorithmic models into healthcare decision-making, a crucial advancement in patient care. However, despite these limitations, it is essential to emphasize the significant sample size of our study, comprising 9,352 individuals meeting the inclusion criteria. This sample size is significantly larger than those of previous studies, whether for validating prediction models or developing new machine-learning-based models. It provides robustness and statistical power, enhancing the reliability and generalizability of our study results.

Conclusions

This study demonstrated that the BCPI model, constructed using CBC/DC data and ML techniques, performs comparably to the widely used CURB-65 in predicting outcomes for patients with pneumonia in the emergency department. Furthermore, by incorporating the CURB-65 score with the BCPI model into a Cox regression model, we have demonstrated enhanced prediction capabilities, particularly for low-risk patients. Given its simple parameters and straightforward training process, the Cox regression model holds promise as a more effective prediction tool for categorizing patients with pneumonia in the emergency room.

Availability of data and materials

The data used in this study, sourced from the Chang Gung Research Database at Chang Gung Memorial Hospital, are not publicly accessible owing to usage limitations and licensing agreements. However, individuals interested in accessing the data can request it directly from the authors of the study, provided they obtain the necessary permissions from the Chang Gung Research Database and Chang Gung Memorial Hospital.

Abbreviations

ML:

Machine learning

BCPI:

Blood culture prediction index

CBC:

Complete blood count

DC:

Differential leukocyte count

ED:

Emergency department

AUC:

Area under curve

ROC:

Receiver operating characteristics

PSI:

Pneumonia severity index

CAP:

Community-acquired pneumonia

HAP:

Hospital-acquired pneumonia

VAP:

Ventilator-associated pneumonia

CRP:

C-reactive protein

PCT:

Procalcitonin

CGMH:

Chang Gung Memorial Hospital

CGRD:

Chang Gung Research Database

BUN:

Blood urea nitrogen

GCS:

Glasgow Coma Scale

ATS:

American Thoracic Society

BTS:

British Thoracic Society

NICE:

National Institute for Health and Care Excellence

NLR:

Neutrophil-to-lymphocyte ratio

References

  1. Peyrani P, Mandell L, Torres A, Tillotson GS. The burden of community-acquired bacterial pneumonia in the era of antibiotic resistance. Expert Rev Respir Med. 2019;13(2):139–52.

    Article  CAS  PubMed  Google Scholar 

  2. Musher DM, Thorner AR. Community-acquired pneumonia. N Engl J Med. 2014;371(17):1619–28.

    Article  CAS  PubMed  Google Scholar 

  3. File TM. Community-acquired pneumonia. Lancet. 2003;362(9400):1991–2001.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Lim WS, van der Eerden MM, Laing R, Boersma WG, Karalus N, Town GI, Lewis SA, Macfarlane JT. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377–82.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Bartlett JG, Dowell SF, Mandell LA, File TM Jr, Musher DM, Fine MJ. Practice guidelines for the management of community-acquired pneumonia in adults. Infectious Diseases Society of America. Clin Infect Dis. 2000;31(2):347–82.

    Article  CAS  PubMed  Google Scholar 

  6. Alavi-Moghaddam M, Bakhshi H, Rezaei B, Khashayar P. Pneumonia severity index compared to CURB-65 in predicting the outcome of community acquired pneumonia among patients referred to an Iranian emergency department: a prospective survey. Braz J Infect Dis. 2013;17(2):179–83.

    Article  PubMed  PubMed Central  Google Scholar 

  7. España PP, Capelastegui A, Gorordo I, Esteban C, Oribe M, Ortega M, Bilbao A, Quintana JM. Development and validation of a clinical prediction rule for severe community-acquired pneumonia. Am J Respir Crit Care Med. 2006;174(11):1249–56.

    Article  PubMed  Google Scholar 

  8. Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, Coley CM, Marrie TJ, Kapoor WN. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336(4):243–50.

    Article  CAS  PubMed  Google Scholar 

  9. Lim WS, Baudouin SV, George RC, Hill AT, Jamieson C, Jeune IL, Macfarlane JT, Read RC, Roberts HJ, Levy ML, et al. BTS guidelines for the management of community acquired pneumonia in adults: update 2009. Thorax. 2009;64(Suppl 3):iii1–55.

    Article  PubMed  Google Scholar 

  10. Metlay JP, Waterer GW, Long AC, Anzueto A, Brozek J, Crothers K, Cooley LA, Dean NC, Fine MJ, Flanders SA, et al. Diagnosis and Treatment of Adults with Community-acquired Pneumonia. An Official Clinical Practice Guideline of the American Thoracic Society and Infectious Diseases Society of America. Am J Respir Crit Care Med. 2019;200(7):e45–67.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Serisier DJ, Williams S, Bowler SD. Australasian respiratory and emergency physicians do not use the pneumonia severity index in community-acquired pneumonia. Respirology. 2013;18(2):291–6.

    Article  PubMed  Google Scholar 

  12. Aujesky D, Fine MJ. The pneumonia severity index: a decade after the initial derivation and validation. Clin Infect Dis. 2008;47(Suppl 3):S133-139.

    Article  PubMed  Google Scholar 

  13. Chalmers JD, Singanayagam A, Akram AR, Mandal P, Short PM, Choudhury G, Wood V, Hill AT. Severity assessment tools for predicting mortality in hospitalised patients with community-acquired pneumonia. Systematic review and meta-analysis. Thorax. 2010;65(10):878–83.

    Article  PubMed  Google Scholar 

  14. Buising KL, Thursky KA, Black JF, MacGregor L, Street AC, Kennedy MP, Brown GV. A prospective comparison of severity scores for identifying patients with severe community acquired pneumonia: reconsidering what is meant by severe pneumonia. Thorax. 2006;61(5):419–24.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Man SY, Lee N, Ip M, Antonio GE, Chau SS, Mak P, Graham CA, Zhang M, Lui G, Chan PK, et al. Prospective comparison of three predictive rules for assessing severity of community-acquired pneumonia in Hong Kong. Thorax. 2007;62(4):348–53.

    Article  PubMed  Google Scholar 

  16. Kang SY, Cha WC, Yoo J, Kim T, Park JH, Yoon H, Hwang SY, Sim MS, Jo IJ, Shin TG. Predicting 30-day mortality of patients with pneumonia in an emergency department setting using machine-learning models. Clin Exp Emerg Med. 2020;7(3):197–205.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Siljan WW, Holter JC, Michelsen AE, Nymo SH, Lauritzen T, Oppen K, Husebye E, Ueland T, Mollnes TE, Aukrust P, et al. Inflammatory biomarkers are associated with aetiology and predict outcomes in community-acquired pneumonia: results of a 5-year follow-up cohort study. ERJ Open Research. 2019;5(1):00014–2019.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Lee JH, Kim J, Kim K, Jo YH, Rhee J, Kim TY, Na SH, Hwang SS. Albumin and C-reactive protein have prognostic significance in patients with community-acquired pneumonia. J Crit Care. 2011;26(3):287–94.

    Article  CAS  PubMed  Google Scholar 

  19. Krüger S, Ewig S, Marre R, Papassotiriou J, Richter K, von Baum H, Suttorp N, Welte T. Procalcitonin predicts patients at low risk of death from community-acquired pneumonia across all CRB-65 classes. Eur Respir J. 2008;31(2):349–55.

    Article  PubMed  Google Scholar 

  20. Wiemken TL, Furmanek S, Mattingly WA, Guinn BE, Cavallazzi RS, Fernandez-Botran R, Wolf LA, English C, Ramirez J. Predicting 30-Day Mortality in Hospitalized Patients with Community-Acquired Pneumonia Using Statistical and Machine Learning Approaches. Univ Louisville J Respir Infect. 2017;1(3):10.

    Google Scholar 

  21. Cilloniz C, Ward L, Mogensen ML, Pericàs JM, Méndez R, Gabarrús A, Ferrer M, Garcia-Vidal C, Menendez R, Torres A. Machine-Learning Model for Mortality Prediction in Patients With Community-Acquired Pneumonia: Development and Validation Study. Chest. 2023;163(1):77–88.

  22. Quah J, Liew CJY, Zou L, Koh XH, Alsuwaigh R, Narayan V, Lu TY, Ngoh C, Wang Z, Koh JZ, et al. Chest radiograph-based artificial intelligence predictive model for mortality in community-acquired pneumonia. BMJ Open Respir Res. 2021;8(1): e001045.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Bae Y, Moon H, Kim S. Predicting the mortality of pneumonia patients visiting the emergencydepartment through machine learning. Eur Resp J. 2018;52(suppl 62):PA2635.

    Google Scholar 

  24. Lien F, Lin HS, Wu YT, Chiueh TS. Bacteremia detection from complete blood count and differential leukocyte count with machine learning: complementary and competitive with C-reactive protein and procalcitonin tests. BMC Infect Dis. 2022;22(1):287.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Shao SC, Chan YY, Kao Yang YH, Lin SJ, Hung MJ, Chien RN, Lai CC, Lai EC. The Chang Gung Research Database-A multi-institutional electronic medical records database for real-world epidemiological studies in Taiwan. Pharmacoepidemiol Drug Saf. 2019;28(5):593–600.

    Article  PubMed  Google Scholar 

  26. Qureshi KN, Hodkinson HM. Evaluation of a ten-question mental test in the institutionalized elderly. Age Ageing. 1974;3(3):152–7.

    Article  CAS  PubMed  Google Scholar 

  27. Zheng N, Zhu D, Han Y. Procalcitonin and C-reactive protein perform better than the neutrophil/lymphocyte count ratio in evaluating hospital acquired pneumonia. BMC Pulm Med. 2020;20(1):166.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Hedlund J, Hansson LO. Procalcitonin and C-Reactive Protein Levels in Community-Acquired Pneumonia: Correlation with Etiology and Prognosis. Infection. 2000;28(2):68–73.

    Article  CAS  PubMed  Google Scholar 

  29. Bafadhel M, Clark TW, Reid C, Medina MJ, Batham S, Barer MR, Nicholson KG, Brightling CE. Procalcitonin and C-reactive protein in hospitalized adult patients with community-acquired pneumonia or exacerbation of asthma or COPD. Chest. 2011;139(6):1410–8.

    Article  CAS  PubMed  Google Scholar 

  30. Guyon I, Weston J, Barnhill S, Vapnik V. Gene Selection for Cancer Classification using Support Vector Machines. Machine Learning. 2002;46(1):389–422.

    Article  Google Scholar 

  31. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.

    Article  CAS  PubMed  Google Scholar 

  32. Hosmer DW Jr., Lemeshow S, Sturdivant RX. Applied logistic regression. Third edition. Hoboken (N.J.): Wiley; 2013.

    Book  Google Scholar 

  33. Capelastegui A, España PP, Quintana JM, Areitio I, Gorordo I, Egurrola M, Bilbao A. Validation of a predictive rule for the management of community-acquired pneumonia. Eur Respir J. 2006;27(1):151–7.

    Article  CAS  PubMed  Google Scholar 

  34. Wu TY, Majeed A, Kuo KN. An overview of the healthcare system in Taiwan. London J Prim Care (Abingdon). 2010;3(2):115–9.

    Article  PubMed  Google Scholar 

  35. Aujesky D, Auble TE, Yealy DM, Stone RA, Obrosky DS, Meehan TP, Graff LG, Fine JM, Fine MJ. Prospective comparison of three validated prediction rules for prognosis in community-acquired pneumonia. Am J Med. 2005;118(4):384–92.

    Article  PubMed  Google Scholar 

  36. Barlow G, Nathwani D, Davey P. The CURB65 pneumonia severity score outperforms generic sepsis and early warning scores in predicting mortality in community-acquired pneumonia. Thorax. 2007;62(3):253–9.

    Article  PubMed  Google Scholar 

  37. Choudhury G, Chalmers JD, Mandal P, Akram AR, Murray MP, Short P, Singanayagam A, Hill AT. Physician judgement is a crucial adjunct to pneumonia severity scores in low-risk patients. Eur Respir J. 2011;38(3):643–8.

    Article  CAS  PubMed  Google Scholar 

  38. Aliberti S, Ramirez J, Cosentini R, Brambilla AM, Zanaboni AM, Rossetti V, Tarsia P, Peyrani P, Piffer F, Blasi F. Low CURB-65 is of limited value in deciding discharge of patients with community-acquired pneumonia. Respir Med. 2011;105(11):1732–8.

    Article  PubMed  Google Scholar 

  39. Shah B, Ahmed W, Dhobi G, Shah N, Khursheed S, Haq I. Validity of pneumonia severity index and CURB-65 severity scoring systems in community acquired pneumonia in an Indian setting. Indian J Chest Dis Allied Sci. 2010;52:9–17.

    Article  PubMed  Google Scholar 

  40. Song M, Graubard BI, Rabkin CS, Engels EA. Neutrophil-to-lymphocyte ratio and mortality in the United States general population. Sci Rep. 2021;11(1):464.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. de Jager CPC, van Wijk PTL, Mathoera RB, de Jongh-Leuvenink J, van der Poll T, Wever PC. Lymphocytopenia and neutrophil-lymphocyte count ratio predict bacteremia better than conventional infection markers in an emergency care unit. Crit Care. 2010;14(5):R192.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Kuikel S, Pathak N, Poudel S, Thapa S, Bhattarai SL, Chaudhary G, Pandey KR. Neutrophil-lymphocyte ratio as a predictor of adverse outcome in patients with community-acquired pneumonia: A systematic review. Health Sci Rep. 2022;5(3): e630.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Zhu JS, Ge P, Jiang C, Zhang Y, Li X, Zhao Z, Zhang L, Duong TQ. Deep-learning artificial intelligence analysis of clinical variables predicts mortality in COVID-19 patients. J Am Coll Emerg Physicians Open. 2020;1(6):1364–73.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Ma X, Ng M, Xu S, Xu Z, Qiu H, Liu Y, Lyu J, You J, Zhao P, Wang S, et al. Development and validation of prognosis model of mortality risk in patients with COVID-19. Epidemiol Infect. 2020;148: e168.

    Article  CAS  PubMed  Google Scholar 

  45. Smith RP, Lipworth BJ, Cree IA, Spiers EM, Winter JH. C-reactive protein. A clinical marker in community-acquired pneumonia. Chest. 1995;108(5):1288–91.

    Article  CAS  PubMed  Google Scholar 

  46. Travlos A, Bakakos A, Vlachos KF, Rovina N, Koulouris N, Bakakos P. C-Reactive Protein as a Predictor of Survival and Length of Hospital Stay in Community-Acquired Pneumonia. J Pers Med. 2022;12(10):1710.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Agapakis DI, Tsantilas D, Psarris P, Massa EV, Kotsaftis P, Tziomalos K, Hatzitolios AI. Coagulation and inflammation biomarkers may help predict the severity of community-acquired pneumonia. Respirology. 2010;15(5):796–803.

    Article  PubMed  Google Scholar 

  48. Nseir W, Farah R, Mograbi J, Makhoul N. Impact of serum C-reactive protein measurements in the first 2 days on the 30-day mortality in hospitalized patients with severe community-acquired pneumonia: a cohort study. J Crit Care. 2013;28(3):291–5.

    Article  CAS  PubMed  Google Scholar 

  49. Chalmers JD, Singanayagam A, Hill AT. C-reactive protein is an independent predictor of severity in community-acquired pneumonia. Am J Med. 2008;121(3):219–25.

    Article  CAS  PubMed  Google Scholar 

  50. Saldías-Peñafiel F, Salinas-Rossel G, Farcas-Oksenberg K, Reyes-Sánchez A, Díaz-Patiño O. Immunocompetent adults hospitalized for a community-acquired pneumonia: Serum C-reactive protein as a prognostic marker. Rev Med Chil. 2019;147(8):983–92.

    Article  PubMed  Google Scholar 

  51. Keramat F, Ghasemi Basir HR, Abdoli E, Shafiei Aghdam A, Poorolajal J. Association of serum procalcitonin and C-reactive protein levels with CURB-65 criteria among patients with community-acquired pneumonia. Int J Gen Med. 2018;11:217–23.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Lou AH, Elnenaei MO, Sadek I, Thompson S, Crocker BD, Nassar B. Evaluation of the impact of a total automation system in a large core laboratory on turnaround time. Clin Biochem. 2016;49(16–17):1254–8.

    Article  CAS  PubMed  Google Scholar 

  53. Eccles S, Pincus C, Higgins B, Woodhead M. Diagnosis and management of community and hospital acquired pneumonia in adults: summary of NICE guidance. Bmj. 2014;349:g6722.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors are grateful to Chang Gung Memorial Hospital for providing access to the Chang Gung Research Database. We also acknowledge the Health Information and Epidemiology Laboratory and the Department of Internal Medicine at Chiayi Chang Gung Memorial Hospital for their valuable administrative assistance.

Funding

No funding was received from any specific source for this study.

Author information

Authors and Affiliations

  1. Department of Internal Medicine, Chang Gung Memorial Hospital, No. 6, W. Sec., Jiapu Rd., Puzih, Chiayi County, 613, Taiwan

    Yin-Ting Lin

  2. Division of Allergy, Immunology and Rheumatology, Department of Internal Medicine, Chang Gung Memorial Hospital, No. 6, W. Sec., Jiapu Rd, Puzih, Chiayi County, 613, Taiwan

    Ko-Ming Lin & Frank Lien

  3. Department of Emergency Medicine, Chang Gung Memorial Hospital, No. 6, W. Sec., Jiapu Rd., Puzih, Chiayi County, 613, Taiwan

    Kai-Hsiang Wu

  4. Department of Nursing, Chang Gung University of Science and Technology, Chiayi Campus, Chiayi, Taiwan

    Kai-Hsiang Wu

  5. Graduate Institute of Clinical Medical Sciences, College of Medicine, Chang Gung University, Taoyuan, Taiwan

    Kai-Hsiang Wu

Authors
  1. Yin-Ting Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ko-Ming Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai-Hsiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Frank Lien
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The study was initiated by YTL and FL. FL, KML, and KHW designed the study, while FL and KHW performed the analyses. YTL drafted the manuscript. All authors contributed resources to the study. The validation, manuscript revision, and final approval were conducted collectively by all authors. FL and KHW serve as co-corresponding authors, each contributing equally to the study.

Corresponding authors

Correspondence to Kai-Hsiang Wu or Frank Lien.

Ethics declarations

Ethics approval and consent to participate

The Institutional Review Board of the Chang Gung Medical Foundation, affiliated with the Chang Gung Memorial Hospital, approved the study and waived the requirement for informed consent (IRB No. 202201120B0C101).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, YT., Lin, KM., Wu, KH. et al. Enhancing pneumonia prognosis in the emergency department: a novel machine learning approach using complete blood count and differential leukocyte count combined with CURB-65 score. BMC Med Inform Decis Mak 24, 118 (2024). https://0-doi-org.brum.beds.ac.uk/10.1186/s12911-024-02523-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12911-024-02523-1

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us