Abstract
Background The SHARPEN score was developed to predict in-hospital mortality in patients hospitalized for infective endocarditis (IE), undergoing or not undergoing cardiac surgery. A comparison with other available scores has not yet been carried out.
Objective To evaluate the performance of the SHARPEN score in predicting in-hospital mortality in patients hospitalized for IE undergoing cardiac surgery and compare it with that of both nonspecific and IE-specific surgical scores.
Methods Retrospective cohort study including all admissions of patients ≥18 years who underwent cardiac surgery due to active IE (modified Duke criteria) at a tertiary care university hospital between 2007 and 2016. The SHARPEN score was compared to the EuroSCORE, EuroSCORE II, STS-IE, PALSUSE, AEPEI, EndoSCORE and RISK-E scores. Differences P<0.05 were considered statistically significant.
Results A total of 105 hospitalizations of 101 patients (mean age 57.4±14.6 years; 75.2% male) were included. The median SHARPEN score was 11 (9-13) points. The observed in-hospital mortality was 29.5%. There was no statistically significant difference in observed vs. estimated mortality (P = 0.147), with an area under the ROC curve of 0.66 (P = 0.008). In comparison with the other scores, no difference was observed in discriminative ability. The statistics of the SHARPEN score at a cutoff >10 points — positive predictive value (PPV): 38.1%, 95%CI:30.4-46.6; negative predictive value (NPV): 80.0%, 95%CI:69.8-87.4; and accuracy: 58.1%, 95%CI:48.1-67.6 — showed overlapping 95%CIs, indicating no significant difference between scores.
Conclusions The SHARPEN score did not present parameters with a significant difference in relation to the other scores analyzed; despite the easy obtainment of its few variables, it has limited applicability in clinical practice, like other existing scores.
Endocarditis; Cardiac Surgical Procedures; Hospital Mortality
Introduction
Despite advances in its medical and surgical treatment, infective endocarditis (IE) remains associated with severe complications and high mortality. The presentation and course of IE are highly variable, depending on host factors (preexisting heart disease, prosthetic valve, implantable cardiac device), the causative organism, and the adequacy of treatment (antibiotics, surgery). The interaction of these factors results in an in-hospital mortality rate for patients with IE ranging from 15 to 30%. 1 Surgical treatment is required in approximately half of patients with IE due to severe complications. Reasons for considering surgery early in the active phase (while the patient is still receiving antibiotic treatment) are to avoid progressive heart failure and irreversible structural damage caused by severe infection, as well as to prevent systemic embolization. On the other hand, surgical therapy during the active phase of IE is associated with significant risk. 2
Prognostic scores are a reasonable estimate of the risk of death, which is important in clinical decision-making regarding indications for surgery. Estimates are needed to inform patients and their families about surgical risks, and risk stratification allows for a fair comparison of cardiac surgery outcomes, so that surgeons and hospitals treating high-risk patients do not appear worse off than others. 3 The performance of traditional surgical scores (EuroSCORE and EuroSCORE II) and IE-specific scores (STS-IE, PALSUSE, AEPEI, EndoSCORE, RISK-E) in predicting surgical risk in patients undergoing cardiac surgery for IE was recently evaluated in a Brazilian cohort. 4 In that study, the best performer in terms of predicting mortality risk was the EuroSCORE when taking both discrimination power and calibration (observed-to-estimated mortality ratio) into account. However, previous work showed conflicting results, with the EuroSCORE underperforming compared to an IE-specific score (STS-IE). 5 These differences in the literature support the hypothesis that, to date, no single score has proven to be ideal to identify patients with IE at greater risk for in-hospital mortality.
The SHARPEN score was developed to predict in-hospital mortality in hospitalizations secondary to IE in patients undergoing or not undergoing cardiac surgery. 6 Unlike other scores, the SHARPEN score is composed of clinical variables that are readily obtained, which could facilitate its use in clinical practice. Analysis of SHARPEN in a Brazilian cohort in a tertiary care center showed performance similar to that described in the original cohort. 7 Within this context, the objective of the present study was to evaluate the SHARPEN score as a predictor of in-hospital mortality in hospitalized patients with IE who underwent cardiac surgery, and to compare it with traditional and IE-specific surgical scores – an analysis not performed in the literature yet.
Methods
Retrospective cohort study including all admissions between 2007 and 2016 of patients aged ≥18 years who underwent cardiac surgery due to active IE at Hospital de Clínicas de Porto Alegre (HCPA), a tertiary public teaching hospital in southern Brazil. Only patients with definitive IE, diagnosed according to the modified Duke criteria, 8 were included; those whose electronic medical records were unavailable were excluded. Patients were identified from the surgical booking system and by a keyword search in the electronic medical record system of HCPA. The present study was approved by the institutional Research Ethics Committee (protocol no. 16-0632).
The SHARPEN score 6 was calculated for each IE admission. Depending on the calculated score, admissions were classified as low- (2 to 6 points), moderate- (7 to 10 points), or high-risk (11 to 20 points). The parameters of interest were:
-
S ystolic blood pressure <90 mmHg at presentation: 3 points;
-
H eart failure during hospitalization (Framingham criteria 9 ): 2 points;
-
A ge: <50 years: 2 points; 50-65 years: 4 points; >65 years: 6 points;
-
R aised serum creatinine at admission (>2.26 mg/dL): 2 points;
-
P neumonia (≥48 hours after admission): 2 points;
-
E levated C-reactive protein (peak >200 mg/L during hospitalization): 2 points;
-
N on-intravenous drug abuser: 3 points.
The performance of the SHARPEN score was compared with that of the logistic EuroSCORE 10 and EuroSCORE II, 11 as well as the IE-specific STS-IE, 12 PALSUSE, 13 AEPEI, 14 EndoSCORE, 15 and RISK-E 16 scores. Any death during hospitalization, regardless of length of stay, was defined as in-hospital mortality. Creatinine clearance (CrCl) was estimated using the Cockcroft–Gault formula. 17
Preoperative critical status was defined as the presence of one of the following during the same hospitalization: ventricular tachycardia/fibrillation or aborted sudden death, cardiopulmonary resuscitation, mechanical ventilation before induction of anesthesia, administration of inotropic agents, use of intra-aortic balloon pump/ventricular assist device before induction of anesthesia, or acute kidney injury (anuria or oliguria [urinary output <10 mL/h]). 11 Active IE (still on antibiotics at the time of surgery), chronic lung disease, extracardiac arteriopathy, reduced mobility (severe mobility impairment secondary to neuro-musculoskeletal dysfunction), recent myocardial infarction (≤ 90 days), severe pulmonary artery hypertension (systolic pulmonary artery pressure >55 mmHg), severe kidney injury (CrCl <50 mL/min), and urgency of surgery were also defined according to the EuroSCORE II criteria. 11
Statistical analysis
Data were collected directly from patients’ electronic medical records and analyzed using IBM SPSS 21.0, MedCalc 12.5, and OpenEpi 3.01.16 software. Qualitative data were described as absolute and relative frequencies; mean (standard deviation) or median (interquartile range, IQR) were used for quantitative data as appropriate, depending on the normality of distribution, as determined by the Shapiro-Wilk test. In-hospital mortality was compared between groups using the chi-square test or Fisher’s exact test, as appropriate. The optimal cut-off point for continuous scores was defined using the highest Youden’s J index, as calculated by the equation “(sensitivity + specificity) – 1”. Calibration (expressed as the observed-to-estimated [O/E] mortality ratio, i.e ., the standardized mortality ratio [SMR]) and the discriminative ability (expressed by the area under the receiver operating characteristic curve [AUC-ROC]) of the scores were evaluated. The mid-p exact test with Miettinen’s modification was used to calculate the SMR with a 95% confidence interval (CI). AUC-ROC comparisons were performed using the DeLong test. P-values <0.05 were considered significant.
Results
We studied 105 hospitalizations of 101 patients (four patients had two hospitalizations each) for active IE, who underwent cardiac surgery, between 2007 and 2016. Of the 107 hospitalizations initially retrieved, two (1.9%) were excluded due to unavailability or inaccessibility to the electronic medical record. Characteristics of the sample are described in Table 1 . Hemodialysis before surgery was performed in 22 hospitalizations (21.0%): 14 (13.3%) for chronic kidney disease (CKD), six (5.7%) for acute kidney injury, and two (1.9%) for acute-on-CKD.
The leading indication for surgery was heart failure (n = 49; 46.7%); the main procedures performed were mechanical aortic valve replacement (n = 24; 22.9%), biological aortic valve replacement (n = 22; 21.0%), and biological mitral valve replacement (n = 22; 21.0%). The median (IQR) time between prescription of antibiotics and surgery was 13 (6-22) days.
The median (IQR) SHARPEN score was 11 (9-13) points. The prevalence of each score component is described in detail in Table 2 . Overall, 10 admissions (9.5%) were characterized as low-risk, 40 (38.1%) as moderate-risk, and 55 (52.4%) as high-risk.
The observed in-hospital mortality was 29.5% (95%CI: 20.8-38.2%); 20.0% in admissions stratified as low or moderate risk and 38.1% in high-risk admissions (P = 0.068). Low- and moderate-risk admissions were pooled for this analysis due to the limited representativeness of the former.
There was no statistically significant difference in observed vs. estimated mortality in the overall analysis, nor in moderate-risk hospitalizations. Conversely, the observed mortality for low-risk hospitalizations was higher than estimated, while that of high-risk hospitalizations was lower than estimated ( Figure 1 ). There was wide variability in estimated mortality among the analyzed scores ( Figure 2 ). Besides the SHARPEN score, only EuroSCORE, PALSUSE, and RISK-E did not present a statistically significant difference between observed and estimated mortality ( Table 3 ).
– Observed/estimated in-hospital mortality and overall mortality by SHARPEN risk categories.
– Observed and estimated in-hospital mortality according to the tested scores. *Observed mortality was 29.5%, except for the RISK-E, which was 29.0% (five cases of pulmonary/tricuspid IE were excluded, as they are not included in the analysis of this score). Error bars denote the 95% CI.
The AUC-ROC for the SHARPEN score was 0.66 (95%CI: 0.54-0.79; P = 0.008) ( Figure 3 ). In comparison with the other scores ( Table 3 ), there was no significant difference in discrimination, with only a trend for discriminative ability to be lower than that of the logistic EuroSCORE (P = 0.098) and EndoSCORE (P = 0.110).
Statistics for the SHARPEN score, and for the other scores analyzed, are shown in the Central Figure. An increase in positive predictive value (PPV) and in accuracy can be observed when adopting a cutoff of >12 points, defined according to Youden’s J statistic, instead of the threshold used in the original study (>10 points). Using the original threshold, mortality was significantly higher (51.7 vs. 21.0%, P = 0.004). However, the CIs of these statistics – as well as those of the negative predictive value (NPV) – overlap, indicating absence of a statistically significant difference between them.
Discussion
In this cohort of patients, the SHARPEN score was calibrated (P = 0.147), with no statistically significant difference in discriminative power (AUC-ROC = 0.66; P = 0.008) in relation to the other tested scores. Likewise, the PPV, NPV, and accuracy of SHARPEN were not significantly different from those of the other scores included in this analysis. Compared to the original cohort, 6 there was a difference in O/E mortality ratio between the low- and high-risk groups, especially in the former (O/E: 10.3; P = 0.003). This difference may be explained by the fact that, while in the original study the proportion of patients undergoing cardiac surgery was 38.3%, our cohort was composed exclusively of surgical patients.
The possibility of using a tool that yields reliable prognostic information based on clinical variables alone is a potential advantage of this new score. In addition, these are objective variables, hence easy to obtain and standardize, which facilitates outcome measurement across different populations. In addition, the SHARPEN score does not include microbiological variables (which may be affected by issues such as difficult confirmation or slow-growing pathogens), unlike other STS-IEs, such as the STS-IE, 12 PALSUSE, 13 EndoSCORE, 15 and RISK-E. 16 However, as it incorporates two variables that can only be identified late in the admission (nosocomial pneumonia and peak C-reactive protein >200 mg/L), it may have limited value in early risk estimation.
In the original study cohort including 233 patients with an in-hospital mortality of 23.2%, of which only 51 (21.9%) underwent cardiac surgery, the score showed an AUC-ROC of 0.86 (95%CI: 0.80-0.91). 6 In the only study that validated the SHARPEN score in Brazil, 7 including 179 hospitalizations for IE in a public hospital, with an in-hospital mortality rate of 22.3% and cardiac surgery performed in 68 patients (38.0%), the score showed an AUC-ROC of 0.76 (95%CI: 0.67-0.85); 0.77 when only clinical treatment was performed and 0.72 in those requiring cardiac surgery. In this study, there was no analysis of specific mortality in the high-risk group (>10 points).
A key strength of the present analysis was the evaluation of the score performance in a cohort of exclusively hospitalizations in which cardiac surgery was performed, unlike the original cohort 6 and the subsequent Brazilian study that also evaluated the performance of the score. 7 Furthermore, while in the original study only 26.2% of admissions were categorized as high risk, in the present series they represented 52.4% of the total. On the other hand, while the feared S. aureus was the main etiologic agent in the original cohort (48.1%), in this cohort it was the cause of only 8.6% of hospitalizations. In the present study, we observed a nonsignificant increase in score accuracy when we adopted a higher cutoff point, raising the hypothesis that, perhaps, for high-risk surgical patients, the cutoff point should be higher; this will need to be confirmed in future cohorts.
Unlike previous studies and comparative studies, we chose to analyze the score statistics starting from a defined cutoff point, as we believe this information is more useful in daily clinical practice, and then define the best threshold through the optimal sensitivity and specificity ratio (Youden’s index). Using the AUC-ROC allowed us to determine the overall accuracy of the score based on the analysis of all points in the score. Given a specific score, and the score of a given patient, patients can be placed in low- or high-risk groups, yielding an estimate (within a 95%CI) of the associated mortality.
The present study has limitations. The sample was relatively small and restricted to a single tertiary care center. The long period of analysis means that both clinical and surgical management of these patients may have changed over time. Finally, retrospective data collection can compromise the quality of the data obtained.
Conclusions
Several research groups have been searched for the optimal prognostic score for risk stratification in cardiac surgery in IE, although no one score has proven superior to others for this purpose. 18 The findings obtained in the present cohort corroborate this statement, since none of the tested scores could be considered ideal, with AUC-ROCs considered reasonable at best (0.7 < AUC-ROC ≤ 0.8), with maximum accuracy of 75.2%. Specifically in relation to the SHARPEN score, its properties did not differ significantly from those of the other analyzed scores; despite the easy obtainment of its few variables, it has limited applicability in clinical practice, like other existing scores.
References
-
1 Marques A, Cruz I, Caldeira D, Alegria S, Gomes AC, Broa AL, et al. Risk Factors for In-Hospital Mortality in Infective Endocarditis. Arq Bras Cardiol. 2020;114(1):1-8. doi: 10.36660/abc.20180194.
» https://doi.org/10.36660/abc.20180194 -
2 Habib G, Lancellotti P, Antunes MJ, Bongiorni MG, Casalta JP, Del Zotti F, et al. 2015 ESC Guidelines for the Management of Infective Endocarditis: The Task Force for the Management of Infective Endocarditis of the European Society of Cardiology (ESC). Endorsed by: European Association for Cardio-Thoracic Surgery (EACTS), the European Association of Nuclear Medicine (EANM). Eur Heart J. 2015;36(44):3075-128. doi: 10.1093/eurheartj/ehv319.
» https://doi.org/10.1093/eurheartj/ehv319 -
3 Martins ABB, Lamas CDC. Prognostic Scores for Mortality in Cardiac Surgery for Infective Endocarditis. Arq Bras Cardiol. 2020;114(3):525-9. doi: 10.36660/abc.20200070.
» https://doi.org/10.36660/abc.20200070 -
4 Pivatto F Jr, Bellagamba CCA, Pianca EG, Fernandes FS, Butzke M, Busato SB, et al. Analysis of Risk Scores to Predict Mortality in Patients Undergoing Cardiac Surgery for Endocarditis. Arq Bras Cardiol. 2020;114(3):518-24. doi: 10.36660/abc.20190050.
» https://doi.org/10.36660/abc.20190050 -
5 Wang TK, Oh T, Voss J, Gamble G, Kang N, Pemberton J. Comparison of Contemporary Risk Scores for Predicting Outcomes after Surgery for Active Infective Endocarditis. Heart Vessels. 2015;30(2):227-34. doi: 10.1007/s00380-014-0472-0.
» https://doi.org/10.1007/s00380-014-0472-0 -
6 Chee QZ, Tan YQ, Ngiam JN, Win MT, Shen X, Choo JN, et al. The SHARPEN clinical Risk Score Predicts Mortality in Patients with Infective Endocarditis: An 11-Year Study. Int J Cardiol. 2015;191:273-6. doi: 10.1016/j.ijcard.2015.04.236.
» https://doi.org/10.1016/j.ijcard.2015.04.236 -
7 Alves SG, Pivatto F Jr, Filippini FB, Dannenhauer GP, Miglioranza MH. SHARPEN Score Accurately Predicts In-Hospital Mortality in Infective Endocarditis. Eur J Intern Med. 2021;92:124-7. doi: 10.1016/j.ejim.2021.05.036.
» https://doi.org/10.1016/j.ejim.2021.05.036 -
8 Li JS, Sexton DJ, Mick N, Nettles R, Fowler VG Jr, Ryan T, et al. Proposed Modifications to the Duke Criteria for the Diagnosis of Infective Endocarditis. Clin Infect Dis. 2000;30(4):633-8. doi: 10.1086/313753.
» https://doi.org/10.1086/313753 -
9 McKee PA, Castelli WP, McNamara PM, Kannel WB. The Natural History of Congestive Heart Failure: The Framingham Study. N Engl J Med. 1971;285(26):1441-6. doi: 10.1056/NEJM197112232852601.
» https://doi.org/10.1056/NEJM197112232852601 -
10 Roques F, Nashef SA, Michel P, Gauducheau E, de Vincentiis C, Baudet E, et al. Risk Factors and Outcome in European Cardiac Surgery: Analysis of the EuroSCORE Multinational Database of 19030 Patients. Eur J Cardiothorac Surg. 1999;15(6):816-22. doi: 10.1016/s1010-7940(99)00106-2.
» https://doi.org/10.1016/s1010-7940(99)00106-2 -
11 Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR, et al. EuroSCORE II. Eur J Cardiothorac Surg. 2012;41(4):734-44. doi: 10.1093/ejcts/ezs043.
» https://doi.org/10.1093/ejcts/ezs043 -
12 Gaca JG, Sheng S, Daneshmand MA, O'Brien S, Rankin JS, Brennan JM, et al. Outcomes for Endocarditis Surgery in North America: A Simplified Risk Scoring System. J Thorac Cardiovasc Surg. 2011;141(1):98-106.e1-2. doi: 10.1016/j.jtcvs.2010.09.016.
» https://doi.org/10.1016/j.jtcvs.2010.09.016 -
13 Martínez-Sellés M, Muñoz P, Arnáiz A, Moreno M, Gálvez J, Rodríguez-Roda J, et al. Valve Surgery in Active Infective Endocarditis: A Simple Score to Predict In-Hospital Prognosis. Int J Cardiol. 2014;175(1):133-7. doi: 10.1016/j.ijcard.2014.04.266.
» https://doi.org/10.1016/j.ijcard.2014.04.266 -
14 Gatti G, Perrotti A, Obadia JF, Duval X, Iung B, Alla F, et al. Simple Scoring System to Predict In-Hospital Mortality after Surgery for Infective Endocarditis. J Am Heart Assoc. 2017;6(7):e004806. doi: 10.1161/JAHA.116.004806.
» https://doi.org/10.1161/JAHA.116.004806 -
15 Di Mauro M, Dato GMA, Barili F, Gelsomino S, Santè P, Corte AD, et al. A Predictive Model for Early Mortality after Surgical Treatment of Heart Valve or Prosthesis Infective Endocarditis. The EndoSCORE. Int J Cardiol. 2017;241:97-102. doi: 10.1016/j.ijcard.2017.03.148.
» https://doi.org/10.1016/j.ijcard.2017.03.148 -
16 Olmos C, Vilacosta I, Habib G, Maroto L, Fernández C, López J, et al. Risk Score for Cardiac Surgery in Active Left-Sided Infective Endocarditis. Heart. 2017;103(18):1435-42. doi: 10.1136/heartjnl-2016-311093.
» https://doi.org/10.1136/heartjnl-2016-311093 -
17 Cockcroft DW, Gault MH. Prediction of Creatinine Clearance from Serum Creatinine. Nephron. 1976;16(1):31-41. doi: 10.1159/000180580.
» https://doi.org/10.1159/000180580 -
18 Sousa C, Pinto FJ. Infective Endocarditis: Still More Challenges Than Convictions. Arq Bras Cardiol. 2022;118(5):976-88. doi: 10.36660/abc.20200798.
» https://doi.org/10.36660/abc.20200798
-
Study AssociationThis study is not associated with any thesis or dissertation work.Ethics Approval and Consent to ParticipateThis study was approved by the Ethics Committee on Animal Experiments of the Hospital de Clínicas de Porto Alegre (HCPA) under the protocol number 16-0632.
-
Sources of Funding: There were no external funding sources for this study.
Publication Dates
-
Publication in this collection
20 Oct 2023 -
Date of issue
2023
History
-
Received
2 May 2023 -
Reviewed
23 June 2023 -
Accepted
26 July 2023