Benchmarking and efficiency assessment in intensive care units: a systematic review protocol

ABSTRACT

Objective:  To identify research gaps and propose strategies for enhancing the quality and efficiency of critical care services globally.

Methods:  We will explore and analyze metrics, models, and methodologies for efficiency assessment and benchmarking in intensive care units, including both static and longitudinal approaches. For so, we will comprehensively search three electronic databases (Scopus, Web of Science, and Embase) using predefined keywords combined with Boolean operators. The search will target peer-reviewed studies without time frame restrictions. Duplicates will be removed, and two reviewers will independently assess the eligibility of articles based on predefined inclusion and exclusion criteria. Relevant data will be extracted and organized thematically using a standardized form aligned with PRISMA guidelines. The extracted data will include a study of characteristics, methodologies, and outcomes, enabling structured mapping and synthesis of existing evidence. This review does not require ethical approval.

Results:  The findings will be disseminated through open-access journal publications and national and international conferences presentations. The results will also be shared with key stakeholders, including healthcare professionals, policymakers, and researchers, to foster discussions on improving intensive care unit performance.

Conclusion:  This review will synthesize available evidence on ICU efficiency assessment, contributing to standardization efforts and guiding future research priorities aimed at strengthening critical care management and benchmarking practices worldwide.

Keywords:
Intensive care unit; Efficiency; Benchmarking; Systematic review protocol

INTRODUCTION

Performance assessment and benchmarking in intensive care units (ICUs) are fundamental to improving the quality and effectiveness of critical care. These evaluations are usually conducted over time or by comparing ICUs across sites or regions.(¹,²) Critical care databases, such as ICU registries,(³–⁵) have become a key enabler of ICU evaluation, supporting data-driven management and benchmarking processes.(⁶,⁷) This approach provides a comprehensive understanding of disparities in care and fosters evidence-based improvements in care delivery.(⁸,⁹)

Regarding the results and indicators used to compare critical care units, several measures have been proposed.(¹⁰–¹³) Mortality is widely recognized as a Key Performance Indicator (KPI) and a central quality measure in ICUs, as it directly correlates with other patient outcomes and reflects the effectiveness of medical interventions and the overall quality of care provided.(¹⁴) Length of stay (LOS), in both hospital and ICU, is another commonly used KPI that reflects not only cost and efficiency, but also the efficacy of care processes. While efficacy is typically assessed at specific points in time as discrete outcomes, efficiency reflects a longitudinal perspective, evaluating resource use and outcomes across the entire care episode.(¹²,¹⁵,¹⁶) However, both mortality and LOS are influenced not only by structural factors, patient transfers, case-mix, and severity of illness, but also by pre-ICU care, admission and discharge criteria, end-of-life care practices, and local patterns of ICU utilization.(¹⁷,¹⁸)

To enable fair comparisons across ICUs, risk adjustment is essential. Severity scoring systems such as the Acute Physiology and Chronic Health Evaluation (APACHE), the Simplified Acute Physiology Score (SAPS), and the Mortality Probability Models (MPM) are widely used to estimate mortality risk based on physiological parameters, comorbidities, and admission characteristics.(¹⁹) In addition, locally developed models such as the Australia and New Zealand Risk of Death (ANZROD)(²⁰) and the Intensive Care National Audit and Research Centre (ICNARC) model(²¹) in the United Kingdom play a central role in benchmarking within specific national contexts. These tools enable the calculation of risk-adjusted outcomes, ensuring that benchmarking reflects differences in care quality rather than variations in case-mix.(⁷)

Different methods have been considered for ICU outcome benchmarking. Previous studies compared performance using ranking tables(²²) or funnel plots(²³,²⁴) for single metrics. A dual-metric approach, like the "efficiency matrix",(²⁵,²⁶) combines risk-adjusted mortality and resource use. More recently,(²⁷,²⁸) the average standardized efficiency rate (ASER) has been proposed as an alternative to integrate the standardized mortality ratio (SMR) and the standardized resource use (SRU) into a single parameter, offering a complementary approach to the efficiency matrix. Unsupervised learning methods (e.g., clustering) also show potential to identify ICUs with similar performance.(²²)

Methods from business benchmarking, such as Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA), address multiple inputs and outputs and provide individual targets for efficiency improvement.(²⁹,³⁰) Data Envelopment Analysis and SFA models have been widely used in healthcare systems nationwide, but less in ICU benchmarking.(³¹) Importantly, both DEA and other statistical models rely on risk-adjusted outcomes, especially for metrics like mortality and LOS.

The literature suggests that critical care efficiency should be evaluated using performance indicators, validated in clinical research and observational studies.(⁵,¹⁶) However, challenges exist in measuring these indicators, including variability in definitions and methods, which compromise the replicability of assessments and international benchmarking.(⁵,³²) Benchmarking outcomes in critical care, supported by risk adjustment models and advanced techniques like machine learning, is essential for performance evaluation and continuous quality improvement in ICUs, providing critical guidelines for clinical practice and healthcare management.(²⁴,³³) Moreover, systematic benchmarking and continuous outcome monitoring enable the identification of best practices, which, when incorporated into care processes, can strengthen organizational resilience. This contributes to more effective crisis management, enhances day-to-day performance, and supports an adaptive response to diverse challenges, ensuring consistent delivery of high-quality care.(³⁴)

Given the critical role of efficiency assessment in improving critical care, consolidating existing knowledge on the best metrics, models, and practices for evaluating ICU performance is essential. Despite available approaches, the current state of the art regarding methods for describing ICU efficiency, their advantages and limitations, and appropriate circumstances remains unclear. This consolidation must encompass both qualitative and quantitative dimensions, addressing identified methodological and contextual challenges. In this context, the present study proposes a protocol to conduct a systematic review to explore, identify, and analyze metrics, models, organizational characteristics, and the association between care processes and ICU efficiency, along with the influence of adherence to these processes on efficiency outcomes. Our main objective is to identify research gaps and propose strategies for enhancing the quality and efficiency of critical care services globally. The specific objectives of this review are:

• –

  To describe the indicators used to study efficiency in ICU settings.

• –

  To identify the most suitable models for evaluating ICU efficiency, considering static and longitudinal approaches.

• –

  To examine the organizational variables (structure and process) associated with superior ICU performance.

• –

  To analyze what processes of care were related to efficiency and how adherence to these care processes relates to ICU efficiency.

METHODS

This protocol was structured according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols (PRISMA-P)(³⁵) statement. The systematic review will be conducted according to the guidelines of the PRISMA.(³⁶–³⁸) The protocol was registered in the international database of systematic reviews, PROSPERO (registration number CRD42025630494), on 22 Dec 2024, following the initial literature search, and subsequently updated on 20 Jan 2025.

Context and concept

This systematic review aims to evaluate efficiency and benchmarking practices in ICUs. The following definitions and concepts will guide the scope of the review:

1. Intensive Care Units are specialized hospital units designed to provide comprehensive, continuous care to critically ill patients.(³⁹) These units typically employ advanced monitoring technologies, highly trained personnel, and life-support interventions to manage complex medical conditions.(⁴⁰)

2. Efficiency in ICUs is the optimal utilization of available resources to achieve desired health outcomes while balancing quality and cost-effectiveness in patient care.(⁴¹) It reflects how well inputs (e.g., staff, equipment, beds) are transformed into outputs and outcomes (e.g., survival, functional recovery, patient satisfaction) in a technically sound and sustainable manner.(²⁹)

3. Benchmarking in ICUs involves comparing performance metrics across units, hospitals, or regions to identify the best practices and areas for improvement. These evaluations can include static analyses (assessing performance at a single point in time) and longitudinal analyses (tracking performance over time).(²⁴)

4. Quality is an overarching concept that encompasses not only outcome measures (e.g., mortality and functional recovery) but also process measures (e.g., adherence to evidence-based protocols, which are directly related to efficacy) and the dimension of resource utilization (related to efficiency). This approach is consistent with Donabedian's(⁴²) structure-process-outcome model and with the principles of value-based healthcare, which emphasize achieving the best health outcomes while optimizing resource use. By explicitly framing quality in this multidimensional manner, we ensure conceptual clarity and alignment with contemporary performance assessment frameworks in intensive care.(⁴³)

5. ICU performance indicators: the review emphasizes commonly used indicators, including mortality rates, LOS (both in ICUs and hospitals), adherence to evidence-based protocols, resource utilization, and other patient outcomes.(⁴⁴) These indicators are crucial for assessing the quality, efficiency, and equity of critical care delivery.(²⁴)

6. Performance assessment methodologies: this study explores various approaches for evaluating efficiency and benchmarking in ICUs. These include quantitative techniques such as DEA, econometric models, machine learning, logistic regression, and other advanced statistical methods, as well as qualitative analyses (e.g., Patient-Reported Outcome Measures [PROMs] and Patient-Reported Experience Measures [PREMs]) that incorporate contextual and organizational factors. Integrating these approaches enables a more comprehensive and nuanced understanding of ICU efficiency.(³³,⁴¹,⁴⁵)

This conceptual framework provides the foundation for systematically exploring the metrics, models, methodologies, and variables that impact ICU efficiency. It fosters a deeper understanding of benchmarking practices and their implications for healthcare systems worldwide.

Search strategy

The search strategy was developed to systematically identify studies on efficiency assessment and benchmarking in ICUs. The query combined terms related to ICU settings (e.g., "intensive care*", "icu*", "critical care*", "critical ill*", "critically ill*"), evaluation processes (e.g., "compar*", "evaluat*", "assess*"), performance concepts (e.g., "efficienc*", "benchmark*", "resilien*"), and formal analytical approaches (e.g., "data envelopment analy*", "DEA", "machine learning", "regression", "model*", "SMR", "SRU"). Truncation symbols (*) were used to capture variations in word endings and maximize retrieval.

The query will be restricted to the title, abstract, and keyword fields to enhance precision and ensure that the selected articles addressed the core concepts of the review. Complete search strategies, including syntax adaptations for Scopus, Web of Science, and Embase, are presented in table 1S (Supplementary Material).

Study selection

The study selection process involved four main steps: (i) definition of inclusion and exclusion criteria; (ii) title and abstract screening; (iii) full-text eligibility assessment; and (iv) inclusion of additional studies through backward and forward citation tracking.

Studies will be eligible for inclusion if they assessed ICU efficiency using formal, structured, and replicable methodologies (quantitative or qualitative approaches applying analytical models, efficiency analysis techniques, or robust conceptual frameworks) or if they performed benchmarking of ICUs based on systematic comparisons with explicit criteria. Eligible studies must report metrics, models, or evaluations directly related to ICU performance (operational, clinical, managerial, or organizational). Studies involving healthcare professionals, managers, patients, or stakeholders directly engaged in ICU care or management will also be included, as their participation may influence how efficiency is structured, perceived, and evaluated in different healthcare settings. Additionally, studies analyzing variations in ICU performance at regional, national, or international levels will be included.

Exclusion criteria will include studies conducted outside ICU settings (e.g., intermediate care units, emergency or urgent care services); those exclusively focused on pediatric or neonatal populations; studies lacking sufficient methodological detail (i.e., without a clear description of metrics, models, variables, or analytical procedures); and those addressing only clinical or pharmacological outcomes unrelated to efficiency or benchmarking. Also excluded will be studies based solely on predictive models for individual patient outcomes (e.g., mortality risk, sepsis risk, LOS) unless such models are explicitly applied to compare ICU performance. Studies proposing only recalibration of traditional prognostic models without direct benchmarking application or performing benchmarking exclusively between prognostic models without comparison of ICU performance will also be excluded. Furthermore, non-peer-reviewed publications (e.g., abstracts, theses, white papers, opinion articles, narrative reviews), studies with insufficient information on efficiency or benchmarking processes, and those not published in English (unless a complete translation is available) will be excluded to ensure methodological consistency and accuracy in data extraction and interpretation.

Two independent reviewers will apply these criteria during the screening stages. Disagreements will be resolved through discussion or consultation with a third reviewer.

Data extraction

The screening process will be carried out using the Rayyan software (http://rayyan.qcri.org), a free tool that assists with screening titles and abstracts.(⁴⁶) Additionally, the ASReview software will implement active learning during the triage phase. This approach leverages machine learning algorithms to prioritize studies for manual review, thereby optimizing the screening process.(⁴⁷) Two independent reviewers will conduct the initial screening, evaluating the titles and abstracts of the retrieved records based on predefined inclusion and exclusion criteria. Selected articles will be thoroughly evaluated for eligibility.

Risk of bias

We will assess the risk of bias of included studies at the study level using an adapted framework informed by the Prediction model Risk of Bias Assessment Tool (PROBAST)(⁴⁸) and the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD)(⁴⁹) guideline, complemented by methodological guidance for observational health services research. The assessment will focus on six domains relevant to ICU efficiency and benchmarking: (i) representativeness of the study population and potential selection bias; (ii) clarity and consistency in the definition of input variables and predictors; (iii) adequacy of outcome measures (e.g., mortality, LOS, resource utilization); (iv) appropriateness of analytical approaches and risk adjustment procedures; (v) completeness and reliability of data sources; and (vi) transparency of reporting and potential selective outcome reporting. Each domain will be rated as low, high, or unclear risk of bias. Studies will be considered to have a high overall risk of bias if any domain is rated as high risk. Two reviewers will independently complete the assessment, with disagreements resolved through discussion or consultation with a third reviewer.

Data mapping

Upon completing data extraction, the collected literature will be thematically organized. This process will serve as the foundation for data mapping, during which key concepts related to ICU efficiency and benchmarking will be identified and structured into a conceptual framework to guide the comprehension of available evidence. This will enable the synthesis of existing findings by comparative analysis and grouping of approaches, highlighting patterns and trends in the literature regarding these assessments’ practices and outcomes. An electronic spreadsheet developed in accordance with the PRISMA guidelines will be used to document the characteristics of included studies and relevant key information. This includes not only general descriptors such as authors, year of publication, country of origin, study design, objectives, sample size, and comparator groups, but also domain-specific variables such as:

• –

  Efficiency models and analytical frameworks (e.g., DEA, SFA, regression-based models, machine learning techniques);

• –

  Input variables (e.g., number of ICU beds, staff-to-patient ratio, availability of medical equipment);

• –

  Output and outcome measures, distinguishing how each study defines efficiency (the technical relationship between inputs and outputs, e.g., SRU, LOS, bed occupancy rates, cost-effectiveness) and/or quality (the nature of the outputs, e.g., SMR, functional outcomes, patient satisfaction, readmission rates), while explicitly avoiding the conflation of resource utilization with outcome measurement;

• –

  Contextual factors (e.g., hospital type, ICU specialization, geographic region, funding model).

The primary findings will be presented in tabular and narrative formats to characterize ICU efficiency and benchmarking practices. The synthesis process will aim to summarize existing research and identify gaps in the literature, particularly in areas requiring further investigation to expand understanding and guide future research and practices in critical care performance assessment.

To ensure these findings’ practical relevance and applicability, this review protocol will incorporate key methodological considerations, including adapting to the specific health context of ICUs, employing case studies, iteratively refining practices, and integrating expert feedback. This approach aims to provide insights for policymakers, healthcare managers, and researchers, ensuring that benchmarking and efficiency evaluation strategies are tailored to the diverse operational contexts of ICUs.

DISCUSSION

This study will contribute to a comprehensive understanding of the methodologies and practices used in ICU efficiency assessment and benchmarking. This comprehensive understanding, derived from the identification and analysis of diverse metrics, models, and variables, is crucial for ensuring high-quality care and optimal resource allocation in complex hospital environments. Previous research, such as that conducted by Soares et al.(¹⁷,⁵⁰) and Zampieri et al.,(⁵¹–⁵³) demonstrates that organizational characteristics, such as the implementation of protocols and the holding of multidisciplinary meetings, are directly associated with reduced mortality and more efficient resource utilization. Furthermore, scoring systems such as the SAPS, APACHE, and SMS-ICU have been widely adopted for assessing patient severity and predicting clinical outcomes, underscoring the importance of integrating clinical and operational data to enhance hospital management.

Contemporary approaches, including machine learning and hybrid methods that integrate DEA with machine learning, are emerging as significant alternatives for ICU performance evaluation. Advanced statistical models, such as linear regressions, mixed models, and machine learning-based methods, such as neural networks and random forests, offer substantial potential for accurately measuring ICU performance,(¹²,¹³,⁵⁴) such as those based on the ICU LOS, which serves as a proxy for allocative efficiency and contributes to benchmarking analysis. Mortality prediction models commonly used in ICUs also play a critical role in risk stratification and in the quality assessment of services provided.(⁷) Additionally, hybrid approaches integrating DEA with neural networks have proven effective for predicting performance and identifying resilience patterns in hospitals and ICUs.(⁵⁴,⁵⁵) Nevertheless, despite their predictive power, advanced machine learning models raise concerns regarding interpretability and clinical trust.(⁵⁶) Balancing predictive performance with explainability remains a key challenge for their implementation in ICU practice.(⁷)

Beyond optimizing efficiency and quality, this review is crucial for understanding how evaluation methodologies can incorporate organizational resilience, an essential attribute for ICUs to adapt to unexpected demands and sustain quality care. According to Salluh et al.,(³⁴) the resilient ICU can adapt to crises or sudden changes in case mix, staffing, or patient volume with minimal impact on clinical outcomes, and can learn from experience to sustain and improve practices over time. Salluh et al.(³⁴) add that resilience can be assessed using the "4S" framework (staff, stuff, space, and systems), complemented by a fifth "S" (science), which reflects the unit's capacity for continuous learning and adaptation.

While this study anticipates significant contributions, it acknowledges potential limitations, including the heterogeneity of available models and metrics and variations in the quality of the included studies. The absence of relevant literature in languages other than English may also limit the representativeness of the findings. This heterogeneity is further compounded by significant variability in the effectiveness of intensive care across regions and contexts, influenced by factors such as critically limited resources, differences in care quality and intervention types, and infrastructure, staff training, and access to advanced technologies.(⁵⁷,⁵⁸)

The findings of this systematic review will be disseminated through publications in academic journals and presentations at national and international conferences. Moreover, the results will be shared with policymakers, health managers, and researchers to foster discussions on how the findings can inform clinical practice, public policy development, and resource allocation. This work is also expected to serve as a foundation for developing new research questions, contributing to methodological and strategic advances in ICU efficiency evaluation.

• Publisher's note
• PROSPERO registration:
  CRD42025630494
• FUNDING
  This work was supported by the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq; 307084/2022-1 and 402971/2023-0 to FLCO; 420096/2023-0 to LSLB; 309546/2025-7 to ITP); the Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro (FAPERJ; E-26/210.858/2024 and E-26/204.540/2024 to ITP; 201.243/2022, and 210.041/2023 to FLCO; E-26/204.520/2024 to LSLB); the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES; Finance Code 001); and the Pontifícia Universidade Católica do Rio de Janeiro.

AVAILABILITY OF DATA AND MATERIALS

The contents are already available.

SUPPLEMENTARY MATERIAL

Supplementary Material

Supplementary Material

REFERENCES

• ¹ Geitona M, Androutsou L, Theodoratou D. Cost estimation of patients admitted to the intensive care unit: a case study of the Teaching University Hospital of Thessaly. J Med Econ. 2010;13(2):179-84.
• ² Tan SS, Bakker J, Hoogendoorn ME, Kapila A, Martin J, Pezzi A, et al. Direct cost analysis of intensive care unit stay in four European countries: applying a standardized costing methodology. Value Health. 2012;15(1):81-6.
• ³ Salluh JI, Quintairos A, Dongelmans DA, Aryal D, Bagshaw S, Beane A, et al.; Linking of Global Intensive Care (LOGIC) and Japanese Intensive care PAtient Database (JIPAD) Working Group. National ICU Registries as Enablers of Clinical Research and Quality Improvement. Crit Care Med. 2024;52(1):125-35.
• ⁴ Soares M, Salluh JI, Zampieri FG, Bozza FA, Kurtz PM. A decade of the ORCHESTRA study: organizational characteristics, patient outcomes, performance and efficiency in critical care. Crit Care Sci. 2024;36:e20240118en.
• ⁵ Dongelmans DA, Pilcher D, Beane A, Soares M, Del Pilar Arias Lopez M, Fernandez A, et al. Linking of global intensive care (LOGIC): an international benchmarking in critical care initiative. J Crit Care. 2020;60:305-10.
• ⁶ Miller RS, Patton M, Graham RM, Hollins D. Outcomes of trauma patients who survive prolonged lengths of stay in the intensive care unit. J Trauma. 2000;48(2):229-34.
• ⁷ Atallah L, Nabian M, Brochini L, Amelung PJ. Machine learning for benchmarking critical care outcomes. Healthc Inform Res. 2023;29(4):301-14.
• ⁸ Friedrich JO, Wilson G, Chant C. Long-term outcomes and clinical predictors of hospital mortality in very long stay intensive care unit patients: a cohort study. Crit Care. 2006;10(2):R59.
• ⁹ Cox CE, Carson SS, Lindquist JH, Olsen MK, Govert JA, Chelluri L; Quality of Life After Mechanical Ventilation in the Aged (QOL-MV) Investigators. Differences in one-year health outcomes and resource utilization by definition of prolonged mechanical ventilation: a prospective cohort study. Crit Care. 2007;11(1):R9.
• ¹⁰ Austin S, Murthy S, Wunsch H, Adhikari NK, Karir V, Rowan K, et al.; International Forum of Acute Care Trialists. Access to urban acute care services in high- vs. middle-income countries: an analysis of seven cities. Intensive Care Med. 2014;40(3):342-52.
• ¹¹ Hicks P, Huckson S, Fenney E, Leggett I, Pilcher D, Litton E. The financial cost of intensive care in Australia: a multicentre registry study. Med J Aust. 2019;211(7):324-5.
• ¹² Peres IT, Hamacher S, Cyrino Oliveira FL, Bozza FA, Salluh JI. Data-driven methodology to predict the ICU length of stay: a multicentre study of 99,492 admissions in 109 Brazilian units. Anaesth Crit Care Pain Med. 2022;41(6):101142.
• ¹³ Peres IT, Ferrari GF, Quintairos A, Bastos LD, Salluh JI. Validation of a new data-driven SLOSR ICU efficiency measure compared to the traditional SRU. Intensive Care Med. 2023;49(12):1546-8.
• ¹⁴ Dziegielewski C, Talarico R, Imsirovic H, Qureshi D, Choudhri Y, Tanuseputro P, et al. Characteristics and resource utilization of high-cost users in the intensive care unit: a population-based cohort study. BMC Health Serv Res. 2021;21(1):1312.
• ¹⁵ Peres IT, Hamacher S, Oliveira FL, Thomé AM, Bozza FA. What factors predict length of stay in the intensive care unit? Systematic review and meta-analysis. J Crit Care. 2020;60:183-94.
• ¹⁶ Peres IT, Hamacher S, Oliveira FL, Bozza FA, Salluh JI. Prediction of intensive care units length of stay: a concise review. Rev Bras Ter Intensiva. 2021;33(2):183-7.
• ¹⁷ Soares M, Bozza FA, Angus DC, Japiassú AM, Viana WN, Costa R, et al. Organizational characteristics, outcomes, and resource use in 78 Brazilian intensive care units: the ORCHESTRA study. Intensive Care Med. 2015;41(12):2149-60.
• ¹⁸ Kurtz P, Peres IT, Soares M, Salluh JI, Bozza FA. Hospital length of stay and 30-day mortality prediction in stroke: a machine learning analysis of 17,000 ICU admissions in Brazil. Neurocrit Care. 2022;37(2 Suppl 2):313-21.
• ¹⁹ Takala J, Moser A, Raj R, Pettilä V, Irincheeva I, Selander T, et al. Variation in severity-adjusted resource use and outcome in intensive care units. Intensive Care Med. 2022;48(1):67-77.
• ²⁰ Paul E, Bailey M, Kasza J, Pilcher D. The ANZROD model: better benchmarking of ICU outcomes and detection of outliers. Crit Care Resusc. 2016;18(1):25-36.
• ²¹ Chen JC, Wang WJ, Wu VC, Huang TM, Lai CY, Ko WJ, et al.; NSARF Study Group. The ICNARC model is predictive of hospital mortality in critically ill patients supported by acute dialysis. Clin Nephrol. 2012;77(5):392-9.
• ²² Verburg IW, de Keizer NF, Holman R, Dongelmans D, de Jonge E, Peek N. Individual and clustered rankability of ICUs according to case-mix-adjusted mortality. Crit Care Med. 2016;44(5):901-9.
• ²³ Verburg IW, Holman R, Peek N, Abu-Hanna A, de Keizer NF. Guidelines on constructing funnel plots for quality indicators: a case study on mortality in intensive care unit patients. Stat Methods Med Res. 2018;27(11):3350-66.
• ²⁴ Salluh JI, Soares M, Keegan MT. Understanding intensive care unit benchmarking. Intensive Care Med. 2017;43(11):1703-7.
• ²⁵ Rothen HU, Stricker K, Einfalt J, Bauer P, Metnitz PG, Moreno RP, et al. Variability in outcome and resource use in intensive care units. Intensive Care Med. 2007;33(8):1329-36.
• ²⁶ Rapoport J, Teres D, Lemeshow S, Gehlbach S. A method for assessing the clinical performance and cost-effectiveness of intensive care units: a multicenter inception cohort study. Crit Care Med. 1994;22(9):1385-91.
• ²⁷ Bastos LS, Wortel SA, de Keizer NF, Bakhshi-Raiez F, Salluh JI, Dongelmans DA, et al. Comparing continuous versus categorical measures to assess and benchmark intensive care unit performance. J Crit Care. 2022;70:154063.
• ²⁸ Wortel SA, de Keizer NF, Abu-Hanna A, Dongelmans DA, Bakhshi-Raiez F. Number of intensivists per bed is associated with efficiency of Dutch intensive care units. J Crit Care. 2021;62:223-9.
• ²⁹ Hollingsworth B, Dawson PJ, Maniadakis N. Efficiency measurement of health care: a review of non-parametric methods and applications. Health Care Manag Sci. 1999;2(3):161-72.
• ³⁰ Kohl S, Schoenfelder J, Fügener A, Brunner JO. The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals. Health Care Manag Sci. 2019;22(2):245-86.
• ³¹ Bahrami MA, Rafiei S, Abedi M, Askari R. Data envelopment analysis for estimating efficiency of intensive care units: a case study in Iran. Int J Health Care Qual Assur. 2018;31(4):276-82.
• ³² de Vos M, Graafmans W, Keesman E, Westert G, van der Voort PH. Quality measurement at intensive care units: which indicators should we use? J Crit Care. 2007;22(4):267-74.
• ³³ Salluh JI, Chiche JD, Reis CE, Soares M. New perspectives to improve critical care benchmarking. Ann Intensive Care. 2018;8(1):17.
• ³⁴ Salluh JI, Kurtz P, Bastos LS, Quintairos A, Zampieri FG, Bozza FA. The resilient intensive care unit. Ann Intensive Care. 2022;12(1):37.
• ³⁵ Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1.
• ³⁶ Moher D, Liberati A, Tetzlaff J, Altman DG, Group TP; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
• ³⁷ Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 200;339:b2700.
• ³⁸ Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372(71):n71.
• ³⁹ Marshall JC, Bosco L, Adhikari NK, Connolly B, Diaz JV, Dorman T, et al. What is an intensive care unit? A report of the task force of the World Federation of Societies of Intensive and Critical Care Medicine. J Crit Care. 2017;37:270-6.
• ⁴⁰ Akdeniz M, Yardımcı B, Kavukcu E. Ethical considerations at the end-of-life care. SAGE Open Med. 2021;9:20503121211000918.
• ⁴¹ Hollingsworth B. The measurement of efficiency and productivity of health care delivery. Health Econ. 2008;17(10):1107-28.
• ⁴² Donabedian A. Evaluating the Quality of Medical Care. Milbak Q. 2005;83(4):691-729.
• ⁴³ Moore L, Lavoie A, Bourgeois G, Lapointe J. Donabedian's structure-process-outcome quality of care model: validation in an integrated trauma system. J Trauma Acute Care Surg. 2015;78(6):1168-75.
• ⁴⁴ Vissers JM, Bertrand JW, De Vries G. A framework for production control in health care organizations. Prod Plann Contr. 2001;12(6):591-604.
• ⁴⁵ Hollingsworth B. Non-parametric and parametric applications measuring efficiency in health care. Health Care Manag Sci. 2003;6(4):203-18.
• ⁴⁶ Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210.
• ⁴⁷ Oami T, Okada Y, Sakuraya M, Fukuda T, Shime N, Nakada TA. Efficiency and workload reduction of semi-automated citation screening software for creating clinical practice guidelines: a prospective observational study. J Epidemiol. 2024;34(8):380-6.
• ⁴⁸ Wolff RF, Moons KG, Riley RD, Whiting PF, Westwood M, Collins GS, et al.; PROBAST Group. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2019;170(1):51-8.
• ⁴⁹ Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162(1):W1-73.
• ⁵⁰ Soares M, Bozza FA, Azevedo LC, Silva UV, Corrêa TD, Colombari F, et al. Effects of organizational characteristics on outcomes and resource use in patients with cancer admitted to intensive care units. J Clin Oncol. 2016;34(27):3315-24.
• ⁵¹ Zampieri FG, Bozza FA, Moralez GM, Mazza DD, Scotti AV, Santino MS, et al. The effects of performance status one week before hospital admission on the outcomes of critically ill patients. Intensive Care Med. 2017;43(1):39-47.
• ⁵² Zampieri FG, Lisboa TC, Correa TD, Bozza FA, Ferez M, Fernandes HS, et al. Role of organizational factors on the ‘weekend effect’ in critically ill patients in Brazil: a retrospective cohort analysis. BMJ Open. 2018;8(1):e018541.
• ⁵³ Zampieri FG, Iwashyna TJ, Viglianti EM, Taniguchi LU, Viana WN, Costa R, et al.; ORCHESTRA Study Investigators. Association of frailty with short-term outcomes, organ support and resource use in critically ill patients. Intensive Care Med. 2018;44(9):1512-20.
• ⁵⁴ Zhong K, Wang Y, Pei J, Tang S, Han Z. Super efficiency SBM-DEA and neural network for performance evaluation. Inf Process Manage. 2021;58(6):102728.
• ⁵⁵ Nandy A, Singh PK. Farm efficiency estimation using a hybrid approach of machine-learning and data envelopment analysis: evidence from rural eastern India. J Clean Prod. 2020;267:122106.
• ⁵⁶ Sridhar GR, Yarabati V, Gumpeny L. Predicting outcomes using neural networks in the intensive care unit. World J Clin Cases. 2025;13(11):100966.
• ⁵⁷ Halpern NA, Pastores SM. Critical care medicine in the United States 2000-2005: an analysis of bed numbers, occupancy rates, payer mix, and costs. Crit Care Med. 2010;38(1):65-71.
• ⁵⁸ Adhikari NK, Fowler RA, Bhagwanjee S, Rubenfeld GD. Critical care and the global burden of critical illness in adults. Lancet. 2010;376(9749):1339-46.

Edited by

• Responsible editor:
  Pedro Póvoa https://orcid.org/0000-0002-7069-7304

Publication Dates

• Publication in this collection
  30 Mar 2026
• Date of issue
  2026

History

• Received
  22 July 2025
• Accepted
  02 Sept 2025