Acessibilidade / Reportar erro

External validation of the parsimonious EuroLung risk models: analysis of the Brazilian Lung Cancer Registry

ABSTRACT

Objective:

The purpose of this study was to assess performance in the Brazilian Lung Cancer Registry Database by using the parsimonious EuroLung risk models for morbidity and mortality.

Methods:

The EuroLung1 and EuroLung2 models were tested and evaluated through calibration (calibration plot, Brier score, and the Hosmer-Lemeshow test) and discrimination (ROC AUCs), in a national multicenter registry of 1,031 patients undergoing anatomic lung resection.

Results:

The evaluation of performance in Brazilian health care facilities utilizing risk-adjustment models, specifically EuroLung1 and EuroLung2, revealed substantial miscalibration, as evidenced by calibration plots and Hosmer-Lemeshow tests in both models. In terms of calibration, EuroLung1 exhibited a calibration plot with overlapping points, characterized by a slope of 1.11 and a Brier score of 0.15; the Hosmer-Lemeshow test yielded a statistically significant p-value of 0.015; and the corresponding ROC AUC was 0.678 (95% CI: 0.636-0.721). The EuroLung2 model displayed better calibration, featuring fewer overlapping points in the calibration plot, with a slope of 1.22, with acceptable discrimination, as indicated by a ROC AUC of 0.756 (95% CI: 0.670-0.842). Both models failed to accurately predict morbidity and mortality outcomes in this specific health care context.

Conclusions:

Discrepancies between the EuroLung model predictions and outcomes in Brazil underscore the need for model refinement and for a probe into inefficiencies in the Brazilian health care system.

Keywords:
Quality of health care; Models, statistical; Public health; Morbidity; Lung neoplasms

INTRODUCTION

In a managed care system, the assessment of care quality within surgical units is crucial. Quality is an abstract concept often measured through various indicators.11 Shahian DM, Edwards FH, Ferraris VA, Haan CK, Rich JB, Normand SL, et al. Quality measurement in adult cardiac surgery: part 1--Conceptual framework and measure selection. Ann Thorac Surg. 2007;83(4 Suppl):S3-S12. https://doi.org/10.1016/j.athoracsur.2007.01.053
https://doi.org/10.1016/j.athoracsur.200...
In thoracic surgery, outcome measures are the main quality indicators. Evaluating the performance of health care providers requires adjusting outcomes for different case mixes across institutions.22 Brunelli A, Rocco G. The comparison of performance between thoracic surgical units. Thorac Surg Clin. 2007;17(3):413-424. https://doi.org/10.1016/j.thorsurg.2007.07.006
https://doi.org/10.1016/j.thorsurg.2007....

To facilitate equitable comparative audits, The European Society of Thoracic Surgeons (ESTS) Database Committee developed risk-adjustment models for morbidity and mortality from a dataset of nearly 50,000 patients.33 Brunelli A, Salati M, Rocco G, Varela G, Van Raemdonck D, Decaluwe H, et al. European risk models for morbidity (EuroLung1) and mortality (EuroLung2) to predict outcome following anatomic lung resections: an analysis from the European Society of Thoracic Surgeons database [published correction appears in Eur J Cardiothorac Surg. 2017 Jun 1;51(6):1212. doi: 10.1093/ejcts/ezx155]. Eur J Cardiothorac Surg. 2017;51(3):490-497. https://doi.org/10.1093/ejcts/ezx155 These models were simplified into the parsimonious EuroLung1 and EuroLung2 versions in 2019.44 Brunelli A, Cicconi S, Decaluwe H, Szanto Z, Falcoz PE. Parsimonious Eurolung risk models to predict cardiopulmonary morbidity and mortality following anatomic lung resections: an updated analysis from the European Society of Thoracic Surgeons database. Eur J Cardiothorac Surg. 2020;57(3):455-461. https://doi.org/10.1093/ejcts/ezz272
https://doi.org/10.1093/ejcts/ezz272...
Those versions offer excellent discrimination capabilities in Europe and are applicable for risk-adjusted performance audits, aiding in quality improvement.

The Brazilian Lung Cancer Registry, a multicenter prospective database, collects data from thoracic procedures at health care facilities in Brazil, supporting quality management. Predictive models like the parsimonious EuroLung risk models facilitate the initial quality assessment and subsequent improvements. Although these models have shown validity in Europe,44 Brunelli A, Cicconi S, Decaluwe H, Szanto Z, Falcoz PE. Parsimonious Eurolung risk models to predict cardiopulmonary morbidity and mortality following anatomic lung resections: an updated analysis from the European Society of Thoracic Surgeons database. Eur J Cardiothorac Surg. 2020;57(3):455-461. https://doi.org/10.1093/ejcts/ezz272
https://doi.org/10.1093/ejcts/ezz272...
they have been shown to have limited discrimination capacity when applied to patients in Canada and Japan.55 Pompili C, Shargall Y, Decaluwe H, Moons J, Chari M, Brunelli A. Risk-adjusted performance evaluation in three academic thoracic surgery units using the Eurolung risk models. Eur J Cardiothorac Surg. 2018;54(1):122-126. https://doi.org/10.1093/ejcts/ezx483
https://doi.org/10.1093/ejcts/ezx483...
,66 Nagoya A, Kanzaki R, Kanou T, Ose N, Funaki S, Minami M, et al. Validation of Eurolung risk models in a Japanese population: a retrospective single-centre analysis of 612 cases. Interact Cardiovasc Thorac Surg. 2019;29(5):722-728. https://doi.org/10.1093/icvts/ivz171
https://doi.org/10.1093/icvts/ivz171...
. To our knowledge, there have been no studies evaluating their applicability in Latin America. This is crucial because of disparities among these populations, including variations in socioeconomic factors and challenges related to diagnosing lung cancer and initiating treatment, often due to barriers to health care access.

The primary objective of this study was to assess the performance of thoracic surgery facilities in Brazil by using the parsimonious EuroLung1 and EuroLung2 risk models within the Brazilian Lung Cancer Registry. A secondary objective was to test the external validity of the parsimonious EuroLung risk models in the Brazilian context.

METHODS

Ethics statement

This study was approved by the local institutional review board (Registration no. 16424413.2.1001.0065). The requirement for informed consent was waived because only anonymized data were used.

Modeling cohort - parsimonious EuroLung1 and EuroLung2 models

In 2017, the ESTS Database Committee published the first models for the prediction of risk after anatomical lung resection (EuroLung1 for cardiopulmonary morbidity and EuroLung2 for 30-day mortality), based on data from approximately 50,000 patients.33 Brunelli A, Salati M, Rocco G, Varela G, Van Raemdonck D, Decaluwe H, et al. European risk models for morbidity (EuroLung1) and mortality (EuroLung2) to predict outcome following anatomic lung resections: an analysis from the European Society of Thoracic Surgeons database [published correction appears in Eur J Cardiothorac Surg. 2017 Jun 1;51(6):1212. doi: 10.1093/ejcts/ezx155]. Eur J Cardiothorac Surg. 2017;51(3):490-497. https://doi.org/10.1093/ejcts/ezx155 A recent update described models that are more parsimonious.44 Brunelli A, Cicconi S, Decaluwe H, Szanto Z, Falcoz PE. Parsimonious Eurolung risk models to predict cardiopulmonary morbidity and mortality following anatomic lung resections: an updated analysis from the European Society of Thoracic Surgeons database. Eur J Cardiothorac Surg. 2020;57(3):455-461. https://doi.org/10.1093/ejcts/ezz272
https://doi.org/10.1093/ejcts/ezz272...
The parsimonious EuroLung models contain five variables for morbidity and six variables for mortality. The two models (EuroLung1 and EuroLung2) contain some common variables associated with morbidity and mortality-age, sex, postoperative FEV1 (ppoFEV1), and thoracotomy-together with some that are specific for either morbidity (extended resection) or mortality (BMI and pneumonectomy).44 Brunelli A, Cicconi S, Decaluwe H, Szanto Z, Falcoz PE. Parsimonious Eurolung risk models to predict cardiopulmonary morbidity and mortality following anatomic lung resections: an updated analysis from the European Society of Thoracic Surgeons database. Eur J Cardiothorac Surg. 2020;57(3):455-461. https://doi.org/10.1093/ejcts/ezz272
https://doi.org/10.1093/ejcts/ezz272...

Cardiopulmonary complications listed in the ESTS database were included as outcome variables.77 Fernandez FG, Falcoz PE, Kozower BD, Salati M, Wright CD, Brunelli A. The Society of Thoracic Surgeons and the European Society of Thoracic Surgeons general thoracic surgery databases: joint standardization of variable definitions and terminology. Ann Thorac Surg. 2015;99(1):368-376. https://doi.org/10.1016/j.athoracsur.2014.05.104
https://doi.org/10.1016/j.athoracsur.201...
Mortality was defined as any death within 30 days after operation or surgical death occurring at any time during the same hospital stay. Extended resection33 Brunelli A, Salati M, Rocco G, Varela G, Van Raemdonck D, Decaluwe H, et al. European risk models for morbidity (EuroLung1) and mortality (EuroLung2) to predict outcome following anatomic lung resections: an analysis from the European Society of Thoracic Surgeons database [published correction appears in Eur J Cardiothorac Surg. 2017 Jun 1;51(6):1212. doi: 10.1093/ejcts/ezx155]. Eur J Cardiothorac Surg. 2017;51(3):490-497. https://doi.org/10.1093/ejcts/ezx155 consisted of chest wall involvement; Pancoast tumors; resection of the atrium, superior vena cava, aorta, diaphragm, or vertebra; bronchial sleeve resection; pleuropneumonectomy; sleeve pneumonectomies; and intrapericardial pneumonectomy.

Aggregate EuroLung2 model

Similar to what was done in the original EuroLung study,77 Fernandez FG, Falcoz PE, Kozower BD, Salati M, Wright CD, Brunelli A. The Society of Thoracic Surgeons and the European Society of Thoracic Surgeons general thoracic surgery databases: joint standardization of variable definitions and terminology. Ann Thorac Surg. 2015;99(1):368-376. https://doi.org/10.1016/j.athoracsur.2014.05.104
https://doi.org/10.1016/j.athoracsur.201...
we tested the aggregate version of the EuroLung2 model to be used as a simple risk stratification tool. using ROC analysis, we found the best cutoff values associated with mortality to be as follows88 Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009.: age > 70 years; ppoFEV1 < 70%; and BMI < 18.5 kg/m2.8 A score of 1 point was assigned to the variables with the smallest odd ratios at logistic regression (age > 70 years and ppoFEV1 < 70%) and proportionally weighting the four other variables44 Brunelli A, Cicconi S, Decaluwe H, Szanto Z, Falcoz PE. Parsimonious Eurolung risk models to predict cardiopulmonary morbidity and mortality following anatomic lung resections: an updated analysis from the European Society of Thoracic Surgeons database. Eur J Cardiothorac Surg. 2020;57(3):455-461. https://doi.org/10.1093/ejcts/ezz272
https://doi.org/10.1093/ejcts/ezz272...
: 2.5 points for male sex, BMI < 18.5 kg/m2, and thoracotomy; and 3 points for pneumonectomy.44 Brunelli A, Cicconi S, Decaluwe H, Szanto Z, Falcoz PE. Parsimonious Eurolung risk models to predict cardiopulmonary morbidity and mortality following anatomic lung resections: an updated analysis from the European Society of Thoracic Surgeons database. Eur J Cardiothorac Surg. 2020;57(3):455-461. https://doi.org/10.1093/ejcts/ezz272
https://doi.org/10.1093/ejcts/ezz272...
Patients were grouped into seven risk classes to evaluate incremental risk of mortality.44 Brunelli A, Cicconi S, Decaluwe H, Szanto Z, Falcoz PE. Parsimonious Eurolung risk models to predict cardiopulmonary morbidity and mortality following anatomic lung resections: an updated analysis from the European Society of Thoracic Surgeons database. Eur J Cardiothorac Surg. 2020;57(3):455-461. https://doi.org/10.1093/ejcts/ezz272
https://doi.org/10.1093/ejcts/ezz272...

Performance evaluation

This study evaluates the performance in Brazilian health care facilities utilizing the EuroLung1 and EuroLung2 risk-adjustment models.44 Brunelli A, Cicconi S, Decaluwe H, Szanto Z, Falcoz PE. Parsimonious Eurolung risk models to predict cardiopulmonary morbidity and mortality following anatomic lung resections: an updated analysis from the European Society of Thoracic Surgeons database. Eur J Cardiothorac Surg. 2020;57(3):455-461. https://doi.org/10.1093/ejcts/ezz272
https://doi.org/10.1093/ejcts/ezz272...
We used a validation cohort from the nationwide multicenter registry known as the Brazilian Lung Cancer Registry. This registry stands as a forward-looking, comprehensive database including patients who have undergone surgical treatment for lung cancer. It involves 12 institutions across five Brazilian states that have provided data related to patients treated between December of 2009 and December of 2022. Our sample comprised 1,031 lung cancer patients who underwent anatomic lung resection during that timeframe, representing 46.25% of all anatomic lung resections cataloged in the Registry. We excluded patients for whom any values pertaining to pivotal variables were missing.

The definitions of variables were derived from the ESTS standardization document.99 Youlden et al. The International Epidemiology of Lung Cancer Geographical Distribution and Secular Trends. Journal of Thoracic Oncology. Volume 3, Number 8, August 2008 https://doi.org/10.1097/JTO.0b013e31818020eb
https://doi.org/10.1097/JTO.0b013e318180...
The goal is to use both risk models as instruments of internal auditing and for quality control in the local context.

Statistical analysis

To test the parsimonious EuroLung1 and EuroLung2 scores, we used the published coefficients for both scores88 Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009. to assess the calibration and discrimination.99 Youlden et al. The International Epidemiology of Lung Cancer Geographical Distribution and Secular Trends. Journal of Thoracic Oncology. Volume 3, Number 8, August 2008 https://doi.org/10.1097/JTO.0b013e31818020eb
https://doi.org/10.1097/JTO.0b013e318180...
,1212 Redondo-Sánchez D, Petrova D, Rodríguez-Barranco M, Fernández-Navarro P, Jiménez-Moleón JJ, Sánchez MJ. Socio-Economic Inequalities in Lung Cancer Outcomes: An Overview of Systematic Reviews. Cancers (Basel). 2022;14(2):398. https://doi.org/10.3390/cancers14020398
https://doi.org/10.3390/cancers14020398...
,1313 Butler CA, Darragh KM, Currie GP, Anderson WJ. Variation in lung cancer survival rates between countries: do differences in data reporting contribute?. Respir Med. 2006;100(9):1642-1646. https://doi.org/10.1016/j.rmed.2005.12.006
https://doi.org/10.1016/j.rmed.2005.12.0...
The logit of the EuroLung1 model was as follows:

−2.852 + 0.021 ×age + 0.472 × male − 0.015 ×ppoFEV1 + 0.662 × thoracotomy + 0.324 × extended resection

The logit of the EuroLung2 model was as follows:

−6.350 + 0.047 × age + 0.889 ×male − 0.055 × BMI − 0.010 × ppoFEV1 + 0.892 × thoracotomy + 0.983 × pneumonectomy

In our assessment, we employed calibration plots, the Brier score, and the Hosmer-Lemeshow test. The calibration plot displays the relationship between observed frequencies and predicted probabilities.88 Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009.,1010 Steyerberg E, Vickers A, Cook N, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. https://doi.org/10.1097/EDE.0b013e3181c30fb2
https://doi.org/10.1097/EDE.0b013e3181c3...
,1111 Royston P, Altman D. External validation of a cox prognostic model: principles and methods. BMC Med Res Methodol. 2013;13:33. https://doi.org/10.1186/1471-2288-13-33
https://doi.org/10.1186/1471-2288-13-33...
The Brier score quantifies the overall disparity between the predicted probability of an event (such as winning) and the actual occurrence of that event.88 Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009.,1010 Steyerberg E, Vickers A, Cook N, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. https://doi.org/10.1097/EDE.0b013e3181c30fb2
https://doi.org/10.1097/EDE.0b013e3181c3...
,1111 Royston P, Altman D. External validation of a cox prognostic model: principles and methods. BMC Med Res Methodol. 2013;13:33. https://doi.org/10.1186/1471-2288-13-33
https://doi.org/10.1186/1471-2288-13-33...
The Hosmer-Lemeshow test divides the study cohort into deciles based on predicted values, comparing the observed rates with the expected rates.88 Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009.,1010 Steyerberg E, Vickers A, Cook N, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. https://doi.org/10.1097/EDE.0b013e3181c30fb2
https://doi.org/10.1097/EDE.0b013e3181c3...
,1111 Royston P, Altman D. External validation of a cox prognostic model: principles and methods. BMC Med Res Methodol. 2013;13:33. https://doi.org/10.1186/1471-2288-13-33
https://doi.org/10.1186/1471-2288-13-33...
Model discrimination was characterized by the ROC AUC.88 Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009.,1010 Steyerberg E, Vickers A, Cook N, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. https://doi.org/10.1097/EDE.0b013e3181c30fb2
https://doi.org/10.1097/EDE.0b013e3181c3...
,1111 Royston P, Altman D. External validation of a cox prognostic model: principles and methods. BMC Med Res Methodol. 2013;13:33. https://doi.org/10.1186/1471-2288-13-33
https://doi.org/10.1186/1471-2288-13-33...

In order to investigate the linear association between the levels of the score for the variable “risk class” and patient mortality (aggregate EuroLung2 model), the Mantel-Haenszel chi-square test (MH χ2) was applied to the data.

Continuous variables are expressed as median and interquartile range, whereas categorical covariates were described as absolute counts and percentages. The 95% confidence intervals are also presented.

Analyses for model development and validation were performed using the R package, version 3.3.3 (R Core Team, 2017) and Stata software, version 15.0 (Stata Corp., College Station, TX, USA). Values of p < 0.05 were considered statistically significant.

RESULTS

Among 1,210 patients who underwent lung resection and were characterized in our database, critical data were missing for 179, and the remaining 1,031 patients were included in further analyses. The characteristics of the included patients are shown in Table 1. Major cardiopulmonary complications occurred in 196 patients (19.0%), and 46 patients (3.8%) died in the hospital or within the first 30 days after the procedure. The observed morbidity rate was higher than that predicted by the EuroLung1 model (19.0% vs. 13.1%). As for mortality, the observed rate was higher than that predicted by the EuroLung2 model (3.8% vs 1.5%). The observed and predicted outcomes in the validation dataset from the EuroLung1 and EuroLung2 models are shown in Tables 2 and 3, respectively.

Table 1
Characteristics of the Brazilian Lung Cancer Registry and European Society of Thoracic Surgeons databases.a

Table 2
Observed and predicted outcomes from the parsimonious EuroLung1 model in the validation cohort (N = 1,031).

Table 3
Observed and predicted outcomes from the parsimonious EuroLung2 model in the validation cohort (N = 1,029).

For the EuroLung1 model, the calibration plot shows some overlap, indicating a lack of perfect calibration. The slope of 1.11 suggests that the model is slightly overestimating probabilities (Figure 1). The Brier score of 0.15 indicates moderate calibration performance, and the p-value of 0.015 from the Hosmer-Lemeshow test suggests that the model is not well calibrated. In addition, the AUC for the EuroLung1 model was 0.678 (95% CI: 0.636-0.721), indicating weak discrimination performance (Figure 2).

Figure 1
Calibration plot for the EuroLung1 and EuroLung2 models (A and B, respectively). The calibration curves highlighted in gray represent the 95% CIs. EuroLung1 model applied to data from 1,031 patients; EuroLung2 model applied to data from 1,029 patients.

Figure 2
Discrimination. ROC curves for the EuroLung1 and EuroLung2 model analyses (A and B, respectively). AUC for the EuroLung1 model analysis = 0.678 (95% CI: 0.636-0.721). AUC for the EuroLung2 model analysis = 0.756 (95% CI: 0.670-0.842). EuroLung1 model applied to data from 1,031 patients; EuroLung2 model applied to data from 1,029 patients.

For the EuroLung2 model, the calibration plot shows less overlap, indicating better calibration (i.e., improved alignment between predicted probabilities and observed outcomes) than that of the EuroLung1 model. The slope of 1.22 further supports that finding, suggesting a closer fit between predicted and observed probabilities (Figure 2). The Brier score of 0.03 indicates good calibration performance, although the Hosmer-Lemeshow test suggested that the model is not well calibrated, given the p-value of 0.044. The EuroLung2 model had acceptable discrimination, as demonstrated by an AUC of 0.756 (95% CI: 0.670-0.842).

Patients were grouped into five risk classes showing incremental risk of mortality, as can be seen in Table 4. There is a statistically significant linear association (p < 0.001; MH χ2 = 6.530, therefore, p < 0.05) between the levels of the score of the aggregate EuroLung2 model and the percentage of mortality of patients. The patients in the lowest risk class had a 3.4% mortality rate, whereas those in the highest risk class had a 28.2% mortality rate. It is noteworthy that the 9.5-12.0 score category was removed from this analysis because it comprised only five cases, which is not sufficient for a reliable prognosis of death.

Table 4
Analysis of linear association between the variables risk class and mortality from the aggregate EuroLung2 model in the sample as a whole (N = 1,205).

DISCUSSION

The external validation assessment of the parsimonious EuroLung1 and EuroLung2 models reveals miscalibration in both. In addition, performance assessments of Brazilian health care facilities using risk-adjustment models like EuroLung1 and EuroLung2 indicate a higher observed mortality and morbidity rate in the Brazilian Lung Cancer Registry than those predicted by the EuroLung risk models. The miscalibration observed in both models indicates the limitations of directly applying them to the Brazilian population without appropriate adjustments, and it emphasizes the need for recalibration or development of locally tailored models to enhance accuracy and improve clinical decision-making. These findings also suggest limitations in the direct application of the EuroLung models to the Brazilian population without suitable modifications, which could potentially highlight the underperformance of health care facilities in Brazil.

The EuroLung risk models represent recent advancements in population-based tools for predicting cardiopulmonary morbidity and mortality following anatomic lung resection, necessitating external validation across diverse populations for generalizability.22 Brunelli A, Rocco G. The comparison of performance between thoracic surgical units. Thorac Surg Clin. 2007;17(3):413-424. https://doi.org/10.1016/j.thorsurg.2007.07.006
https://doi.org/10.1016/j.thorsurg.2007....
,33 Brunelli A, Salati M, Rocco G, Varela G, Van Raemdonck D, Decaluwe H, et al. European risk models for morbidity (EuroLung1) and mortality (EuroLung2) to predict outcome following anatomic lung resections: an analysis from the European Society of Thoracic Surgeons database [published correction appears in Eur J Cardiothorac Surg. 2017 Jun 1;51(6):1212. doi: 10.1093/ejcts/ezx155]. Eur J Cardiothorac Surg. 2017;51(3):490-497. https://doi.org/10.1093/ejcts/ezx155 However, such validation is often hindered by population-specific discrepancies.88 Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009. In the Brazilian cohort, the EuroLung2 model demonstrated acceptable discrimination, as evidenced by a higher AUC value. However, discrepancies in both models probably stem from the exclusion of critical variables in the ESTS model, which are vital in the Brazilian context, such as racial and social factors, along with caseload variations. This observation is consistent with the findings of a study conducted in Japan,66 Nagoya A, Kanzaki R, Kanou T, Ose N, Funaki S, Minami M, et al. Validation of Eurolung risk models in a Japanese population: a retrospective single-centre analysis of 612 cases. Interact Cardiovasc Thorac Surg. 2019;29(5):722-728. https://doi.org/10.1093/icvts/ivz171
https://doi.org/10.1093/icvts/ivz171...
highlighting the predictive limitations of the EuroLung models for morbidity and mortality due to notable baseline differences with the European demographic.66 Nagoya A, Kanzaki R, Kanou T, Ose N, Funaki S, Minami M, et al. Validation of Eurolung risk models in a Japanese population: a retrospective single-centre analysis of 612 cases. Interact Cardiovasc Thorac Surg. 2019;29(5):722-728. https://doi.org/10.1093/icvts/ivz171
https://doi.org/10.1093/icvts/ivz171...
Such omissions might significantly impact the observed underperformance of Brazilian health care facilities. Nonetheless, the discrepancy between the observed and predicted morbidity rates can be attributed to patient-specific factors, which encompass pre-existing comorbidities, socioeconomic conditions, and the disease stage at the time of diagnosis.99 Youlden et al. The International Epidemiology of Lung Cancer Geographical Distribution and Secular Trends. Journal of Thoracic Oncology. Volume 3, Number 8, August 2008 https://doi.org/10.1097/JTO.0b013e31818020eb
https://doi.org/10.1097/JTO.0b013e318180...
,1212 Redondo-Sánchez D, Petrova D, Rodríguez-Barranco M, Fernández-Navarro P, Jiménez-Moleón JJ, Sánchez MJ. Socio-Economic Inequalities in Lung Cancer Outcomes: An Overview of Systematic Reviews. Cancers (Basel). 2022;14(2):398. https://doi.org/10.3390/cancers14020398
https://doi.org/10.3390/cancers14020398...

13 Butler CA, Darragh KM, Currie GP, Anderson WJ. Variation in lung cancer survival rates between countries: do differences in data reporting contribute?. Respir Med. 2006;100(9):1642-1646. https://doi.org/10.1016/j.rmed.2005.12.006
https://doi.org/10.1016/j.rmed.2005.12.0...
-1414 Soares MS, Coltro LM, Leite PHC, Costa PB, Lauricella LL, et al. Evolution of the surgical treatment of lung cancer at a tertiary referral center in Brazil, 2011-2018. J Bras Pneumol. 2020;47(1):e20190426. https://doi.org/10.36416/1806-3756/e20190426
https://doi.org/10.36416/1806-3756/e2019...
In Brazil, a middle-income country, the absence of adequate education regarding disease prevention often results in patients presenting to the health care system with advanced, symptomatic disease,1515 de Sá VK, Coelho JC, Capelozzi VL, de Azevedo SJ. Lung cancer in Brazil: epidemiology and treatment challenges. Lung Cancer (Auckl). 2016;7:141-148. https://doi.org/10.2147/LCTT.S93604
https://doi.org/10.2147/LCTT.S93604...
in contrast to their counterparts in high-income countries. Notably, Knorst et al.1616 Knorst MM, Dienstmann R, Fagundes LP. Delay in the diagnosis and in the surgical treatment of lung cancer [Article in Portuguese] J Pneumol. 2003;29(6):358-364. https://doi.org/10.1590/S0102-35862003000600007
https://doi.org/10.1590/S0102-3586200300...
reported a historical cohort study in which the time from the onset of initial symptoms to the diagnosis of lung cancer in a university hospital in the southern region of Brazil exceeded 20 weeks, whereas the Standing Medical Advisory Committee recommendation is that the interval between symptom onset and treatment should be no longer than 6-8 weeks.

The discrepancy in mortality may be linked to systemic factors, including access to health care services for prevention, timely diagnosis, and treatment.1717 Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71(3):209-249. https://doi.org/10.3322/caac.21660
https://doi.org/10.3322/caac.21660...
In Brazil, over 75% of patients depend exclusively on the Brazilian Unified Health Care System. Despite its goal of providing universal care, the system faces significant challenges related to accessibility, diagnostic delays, treatment availability, and substantial disparities among cancer care facilities concerning diagnostic and treatment technologies.1818 Grabois MF, Oliveira EX, Sá Carvalho M. Access to pediatric cancer care in Brazil mapping origin-destination flows [Article in Portuguese]. Rev Saude Publica. 2013;47(2):368-378. https://doi.org/10.1590/S0034-8910.2013047004305
https://doi.org/10.1590/S0034-8910.20130...
,1919 Ferreira CG. Lung cancer in developing countries: access to molecular testing. Am Soc Clin Oncol Educ Book. 2013;327-331. https://doi.org/10.1200/EdBook_AM.2013.33.327
https://doi.org/10.1200/EdBook_AM.2013.3...
For example, Lista et al.2020 Lista M, Bes FC, Pereira JR, Ikari FK, Nikaedo SM. Excessiva demora no diagnóstico clínico do câncer de pulmão Depende do médico, do paciente ou do sistema? Arq Med Hosp Fac Cienc Med St Casa Sao Paulo. 2008;53(1):6-9. discovered that almost 80% of the initial treatments for lung cancer in Brazil did not take the diagnosis into consideration; only 6.8% of patients received a lung cancer diagnosis within 30 days after experiencing symptoms. Another study conducted among the Brazilian population revealed that 10-18% of lung cancer patients, regardless of their disease stage, did not undergo any cancer treatment due to their poor clinical condition,2121 Costa GJ, Mello MJG, Bergmann A, Ferreira CG, Thuler LCS. Tumor-node-metastasis staging and treatment patterns of 73,167 patients with lung cancer in Brazil. J Bras Pneumol. 2020;46(1):e20180251. https://doi.org/10.1590/1806-3713/e20180251
https://doi.org/10.1590/1806-3713/e20180...
rendering them unable to withstand the risks associated with treatment.

Lung cancer remains a pressing public health concern in Brazil, and as a response to this challenge, the country has implemented a series of public policies aimed at improving surgical treatment outcomes. Over the past decade, Brazil has made significant strides in this area, with initiatives focused on expanding access to early detection, enhancing surgical techniques, and ensuring equitable care for all patients. In addition, strong public health measures in Brazil have led to notable reductions in tobacco consumption in Brazil, setting a valuable precedent for other low- and middle-income countries. National research in Brazil has revealed a nearly 50% reduction in smoking prevalence, aligning with a corresponding decrease in tobacco-related fatalities.2222 Levy D, de Almeida LM, Szklo A. The Brazil SimSmoke policy simulation model: the effect of strong tobacco control policies on smoking prevalence and smoking-attributable deaths in a middle income nation. PLoS Med. 2012;9(11):e1001336. https://doi.org/10.1371/journal.pmed.1001336
https://doi.org/10.1371/journal.pmed.100...
These policies, coupled with efforts to reduce health care disparities, have the potential to revolutionize lung cancer surgery in Brazil, ultimately leading to better patient outcomes and a brighter future in the fight against this devastating disease.

Another reason for the underperformance of Brazilian health care facilities may be related to surgical skills. Therefore, we will examine the data in a more granular manner to gain a deeper understanding of the quality of surgical care at the facilities that could be associated with these outcomes. Subsequently, we will investigate design actions aimed at enhancing improvement factors. Overall, these findings highlight the complex interplay between patient-specific and systemic factors that influence the calibration and performance of risk models in a diverse health care landscape such as that of Brazil. Further research and tailored interventions are essential to bridge these disparities and improve the quality of lung cancer care in the country.

The present study relied on data from the Brazilian Lung Cancer Registry, a prospective multicenter database. The main limitation of the study is the size of the sample, which was small in comparison with the original population from which the models were generated. In addition, the study may simply be underpowered to assess the calibration and discrimination of the risk models. The fact that 46% of the cases were excluded from analysis in both arms because key values were missing raises concerns about the validity of our findings. This significant data gap suggests a potential bias, given that less than half of the facilities contributed meaningful data, limiting the comprehensiveness and reliability of the analysis. Furthermore, the Brazilian Lung Cancer Registry includes 12 institutions in five Brazilian states and does not represent the entire country. However, it is important to note that it stands as the only database related to the surgical treatment of lung cancer in Brazil. Therefore, the findings should be interpreted within the context of the studied population. Moreover, our database initially included mostly patients from the public health care sector, only later including those from the private sector. In the present study, no analyses were carried out separating patients by sector.

The disparities between the EuroLung model predictions and Brazilian patient outcomes highlight the need for model adjustments and signal potential underperformance within the health care system in Brazil, underscoring the importance of investigating contributing factors. The EuroLung2 model showed promising performance in terms of discrimination in the Brazilian cohort, indicating its potential utility. Considering additional variables and exploring machine learning analytics may further enhance the performance of surgical risk prediction models.

ACKNOWLEDGMENTS

We would like to thank all the participants of the Brazilian Lung Cancer Registry.

REFERENCES

  • 1
    Shahian DM, Edwards FH, Ferraris VA, Haan CK, Rich JB, Normand SL, et al. Quality measurement in adult cardiac surgery: part 1--Conceptual framework and measure selection. Ann Thorac Surg. 2007;83(4 Suppl):S3-S12. https://doi.org/10.1016/j.athoracsur.2007.01.053
    » https://doi.org/10.1016/j.athoracsur.2007.01.053
  • 2
    Brunelli A, Rocco G. The comparison of performance between thoracic surgical units. Thorac Surg Clin. 2007;17(3):413-424. https://doi.org/10.1016/j.thorsurg.2007.07.006
    » https://doi.org/10.1016/j.thorsurg.2007.07.006
  • 3
    Brunelli A, Salati M, Rocco G, Varela G, Van Raemdonck D, Decaluwe H, et al. European risk models for morbidity (EuroLung1) and mortality (EuroLung2) to predict outcome following anatomic lung resections: an analysis from the European Society of Thoracic Surgeons database [published correction appears in Eur J Cardiothorac Surg. 2017 Jun 1;51(6):1212. doi: 10.1093/ejcts/ezx155]. Eur J Cardiothorac Surg. 2017;51(3):490-497. https://doi.org/10.1093/ejcts/ezx155
  • 4
    Brunelli A, Cicconi S, Decaluwe H, Szanto Z, Falcoz PE. Parsimonious Eurolung risk models to predict cardiopulmonary morbidity and mortality following anatomic lung resections: an updated analysis from the European Society of Thoracic Surgeons database. Eur J Cardiothorac Surg. 2020;57(3):455-461. https://doi.org/10.1093/ejcts/ezz272
    » https://doi.org/10.1093/ejcts/ezz272
  • 5
    Pompili C, Shargall Y, Decaluwe H, Moons J, Chari M, Brunelli A. Risk-adjusted performance evaluation in three academic thoracic surgery units using the Eurolung risk models. Eur J Cardiothorac Surg. 2018;54(1):122-126. https://doi.org/10.1093/ejcts/ezx483
    » https://doi.org/10.1093/ejcts/ezx483
  • 6
    Nagoya A, Kanzaki R, Kanou T, Ose N, Funaki S, Minami M, et al. Validation of Eurolung risk models in a Japanese population: a retrospective single-centre analysis of 612 cases. Interact Cardiovasc Thorac Surg. 2019;29(5):722-728. https://doi.org/10.1093/icvts/ivz171
    » https://doi.org/10.1093/icvts/ivz171
  • 7
    Fernandez FG, Falcoz PE, Kozower BD, Salati M, Wright CD, Brunelli A. The Society of Thoracic Surgeons and the European Society of Thoracic Surgeons general thoracic surgery databases: joint standardization of variable definitions and terminology. Ann Thorac Surg. 2015;99(1):368-376. https://doi.org/10.1016/j.athoracsur.2014.05.104
    » https://doi.org/10.1016/j.athoracsur.2014.05.104
  • 8
    Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009.
  • 9
    Youlden et al. The International Epidemiology of Lung Cancer Geographical Distribution and Secular Trends. Journal of Thoracic Oncology. Volume 3, Number 8, August 2008 https://doi.org/10.1097/JTO.0b013e31818020eb
    » https://doi.org/10.1097/JTO.0b013e31818020eb
  • 10
    Steyerberg E, Vickers A, Cook N, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. https://doi.org/10.1097/EDE.0b013e3181c30fb2
    » https://doi.org/10.1097/EDE.0b013e3181c30fb2
  • 11
    Royston P, Altman D. External validation of a cox prognostic model: principles and methods. BMC Med Res Methodol. 2013;13:33. https://doi.org/10.1186/1471-2288-13-33
    » https://doi.org/10.1186/1471-2288-13-33
  • 12
    Redondo-Sánchez D, Petrova D, Rodríguez-Barranco M, Fernández-Navarro P, Jiménez-Moleón JJ, Sánchez MJ. Socio-Economic Inequalities in Lung Cancer Outcomes: An Overview of Systematic Reviews. Cancers (Basel). 2022;14(2):398. https://doi.org/10.3390/cancers14020398
    » https://doi.org/10.3390/cancers14020398
  • 13
    Butler CA, Darragh KM, Currie GP, Anderson WJ. Variation in lung cancer survival rates between countries: do differences in data reporting contribute?. Respir Med. 2006;100(9):1642-1646. https://doi.org/10.1016/j.rmed.2005.12.006
    » https://doi.org/10.1016/j.rmed.2005.12.006
  • 14
    Soares MS, Coltro LM, Leite PHC, Costa PB, Lauricella LL, et al. Evolution of the surgical treatment of lung cancer at a tertiary referral center in Brazil, 2011-2018. J Bras Pneumol. 2020;47(1):e20190426. https://doi.org/10.36416/1806-3756/e20190426
    » https://doi.org/10.36416/1806-3756/e20190426
  • 15
    de Sá VK, Coelho JC, Capelozzi VL, de Azevedo SJ. Lung cancer in Brazil: epidemiology and treatment challenges. Lung Cancer (Auckl). 2016;7:141-148. https://doi.org/10.2147/LCTT.S93604
    » https://doi.org/10.2147/LCTT.S93604
  • 16
    Knorst MM, Dienstmann R, Fagundes LP. Delay in the diagnosis and in the surgical treatment of lung cancer [Article in Portuguese] J Pneumol. 2003;29(6):358-364. https://doi.org/10.1590/S0102-35862003000600007
    » https://doi.org/10.1590/S0102-35862003000600007
  • 17
    Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71(3):209-249. https://doi.org/10.3322/caac.21660
    » https://doi.org/10.3322/caac.21660
  • 18
    Grabois MF, Oliveira EX, Sá Carvalho M. Access to pediatric cancer care in Brazil mapping origin-destination flows [Article in Portuguese]. Rev Saude Publica. 2013;47(2):368-378. https://doi.org/10.1590/S0034-8910.2013047004305
    » https://doi.org/10.1590/S0034-8910.2013047004305
  • 19
    Ferreira CG. Lung cancer in developing countries: access to molecular testing. Am Soc Clin Oncol Educ Book. 2013;327-331. https://doi.org/10.1200/EdBook_AM.2013.33.327
    » https://doi.org/10.1200/EdBook_AM.2013.33.327
  • 20
    Lista M, Bes FC, Pereira JR, Ikari FK, Nikaedo SM. Excessiva demora no diagnóstico clínico do câncer de pulmão Depende do médico, do paciente ou do sistema? Arq Med Hosp Fac Cienc Med St Casa Sao Paulo. 2008;53(1):6-9.
  • 21
    Costa GJ, Mello MJG, Bergmann A, Ferreira CG, Thuler LCS. Tumor-node-metastasis staging and treatment patterns of 73,167 patients with lung cancer in Brazil. J Bras Pneumol. 2020;46(1):e20180251. https://doi.org/10.1590/1806-3713/e20180251
    » https://doi.org/10.1590/1806-3713/e20180251
  • 22
    Levy D, de Almeida LM, Szklo A. The Brazil SimSmoke policy simulation model: the effect of strong tobacco control policies on smoking prevalence and smoking-attributable deaths in a middle income nation. PLoS Med. 2012;9(11):e1001336. https://doi.org/10.1371/journal.pmed.1001336
    » https://doi.org/10.1371/journal.pmed.1001336
  • 23
    Erridge SC, Møller H, Price A, Brewster D. International comparisons of survival from lung cancer: pitfalls and warnings. Nat Clin Pract Oncol. 2007;4(10):570-577. https://doi.org/10.1038/ncponc0932
    » https://doi.org/10.1038/ncponc0932
  • MEETING PRESENTATION

    This abstract was presented as a poster at the 31st Annual Conference of the ESTS, in Milano, Italy.

Publication Dates

  • Publication in this collection
    27 Sept 2024
  • Date of issue
    2024

History

  • Received
    16 July 2024
  • Accepted
    24 July 2024
Sociedade Brasileira de Pneumologia e Tisiologia SCS Quadra 1, Bl. K salas 203/204, 70398-900 - Brasília - DF - Brasil, Fone/Fax: 0800 61 6218 ramal 211, (55 61)3245-1030/6218 ramal 211 - São Paulo - SP - Brazil
E-mail: jbp@sbpt.org.br