Environmental performance indicators for health services managementAbstract
Objective To develop environmental performance indicators and validate them with experts in the areas of health service management and/or sustainability.
Methods Applied research, in which indicators were developed based on the 10 objectives of the Global Agenda for Green and Healthy Hospitals of the International Organization for Health Without Harm. Content and construction validation were validated by a panel composed of five experts from the southwestern region of Brazil, using the Delphi technique, between June and November 2024. The responses were demonstrated quantitatively, where indicators with a Content Validation Index (CVI) with a result equal to 1 or 0.8 were considered important.
Results Initially, 44 environmental performance indicators were developed, grouped into eight categories: energy resources, water resources, supply chain/suppliers, transportation system, food management, waste, facilities/infrastructure, and greenhouse gas emissions. These indicators were analyzed by experts, and in the first round, 30 (68.2%) were considered important and 4 (9.1%) were excluded. In the second round, 10 (22,7%) indicators that did not reach consensus in the first round were included along with another 10 indicators suggested by the participants. Of these, 15 were included, and two previous indicators were unified, totaling 44 indicators.
Conclusion 44 environmental performance indicators were developed and validated by experts as important for assisting managers and professionals in decision-making regarding sustainable practices in health services.
Keywords
Health services; Environment; Environmental indicators; Sustainable development
Resumo
Objetivo Desenvolver indicadores de desempenho ambiental e validá-los com especialistas da área.
Métodos Pesquisa aplicada em que foram desenvolvidos indicadores de desempenho ambiental com base nos 10 objetivos da Agenda Global para Hospitais Verdes e Saudáveis da Organização Internacional Saúde Sem Dano. A validação de conteúdo e construto foi realizada por um painel composto por cinco especialistas da região sudoeste do Brasil, utilizando a Técnica Delphi, entre junho e novembro de 2024. As respostas foram analisadas quantitativamente, e os indicadores com Índice de Validação de Conteúdo (IVC) igual a 1 ou 0,8 foram considerados importantes.
Resultados Inicialmente foram desenvolvidos 44 indicadores de desempenho ambiental, agrupados em oito categorias: recursos energéticos, recursos hídricos, cadeia de suprimentos/fornecedores, sistema de transporte, gestão de alimentos, resíduos, instalações/infraestrutura e emissões de gases de efeito estufa. Esses indicadores foram analisados por especialistas e, na primeira rodada, 30 (68,2%) foram considerados importantes e quatro (9,1%) foram excluídos. Na segunda rodada, 10 (22,7%) indicadores que não alcançaram consenso na primeira rodada foram incluídos, juntamente com outros 10 indicadores sugeridos pelos participantes. Destes, 15 foram incluídos e dois indicadores anteriores foram unificados, totalizando 44 indicadores.
Conclusão O total de 44 indicadores de desempenho ambiental foram desenvolvidos e validados por especialistas, considerados importantes para auxiliar gestores e profissionais na tomada de decisões relativas a práticas sustentáveis em serviços de saúde.
Descritores
Enfermagem; Serviços de Saúde; Meio ambiente; Indicadores Ambientais; Desenvolvimento Sustentável
Resumen
Objetivo Elaborar indicadores de rendimiento medioambiental y validarlos con especialistas en la materia.
Métodos Investigación aplicada en la que se elaboraron indicadores de desempeño ambiental basados en los 10 objetivos de la Agenda Global para Hospitales Verdes y Saludables de la organización internacional Salud sin Daño. La validación del contenido y el constructo fueron realizados por un grupo compuesto por cinco especialistas de la región suroeste de Brasil, utilizando el método Delphi, entre junio y noviembre de 2024. Las respuestas se analizaron cuantitativamente, y los indicadores con un Índice de Validez de Contenido (IVC) igual a 1 o 0,8 se consideraron importantes.
Resultados Inicialmente se elaboraron 44 indicadores de desempeño ambiental, agrupados en ocho categorías: recursos energéticos, recursos hídricos, cadena de suministro/proveedores, sistema de transporte, gestión de alimentos, residuos, instalaciones/infraestructura y emisiones de gases de efecto invernadero. Estos indicadores fueron analizados por un grupo de especialistas. En la primera ronda, 30 (68,2 %) fueron consideraros importantes y cuatro (9,1 %) se excluyeron. En la segunda ronda, se incluyeron 10 (22,7 %) indicadores que no alcanzaron consenso en la primera ronda, junto con otros 10 indicadores sugeridos por los participantes. De estos, se incluyeron 15 y se unificaron dos indicadores anteriores, lo que dio un total de 44 indicadores.
Conclusión Se elaboró un total de 44 indicadores de desempeño ambiental, que fueron validados por especialistas, y considerados importantes para ayudar a los gestores y profesionales en la toma de decisiones relacionadas con prácticas sustentables en los servicios de salud.
Descriptores
Enfermería; Servicios de Salud; Ambiente; Indicadores Ambientales; Desarrollo Sostenible
Introduction
Access to information by companies is currently considered a valuable asset. Information can demonstrate more clearly the performance of services and direct the necessary actions towards objectives. From this perspective, the success of an organization depends on how its performance is managed,(1)and this includes health services.
Key Performance Indicators (KPIs) are metrics established to measure an organization’s performance. KPIs provide managers with information about the institution’s performance, identifying strengths and weaknesses, as well as areas that require improvement.(2)The benefits of performance measurement also include the creation of action plans that are aligned with the organization’s objectives of the services.(1)
Concern for the environment has led to increased pressure from stakeholders on companies to change their behavior and focus more on their environmental impacts. This is because investors have considerable influence on companies’ actions and there is a growing number of socially responsible investors.(3)They also believe that it is necessary for companies, including in the health sector, to take care of the environment, be socially responsible and adopt good governance practices.(4)As a result, companies from different segments began to use performance indicators, also related to the environmental aspect, and to share them with stakeholders.
Environmental performance indicators allow for a comparative assessment of an organization’s performance in different environmental aspects.(5)In addition, they allow managers to have a multidimensional view of the current state of sustainability of hospitals and their level of excellence.(6)
Environmental performance indicators at different scales are important for environmental management.(7)This means that organizations that adopt the use of indicators to improve management and monitor these practices will contribute to sustainable development.
Identifying stakeholders in organizations and involving them in the development of these indicators helps to meet their needs, allowing potential barriers to their implementation to be identified and addressed.(8) Stakeholders drive companies’ strategy in relation to KPIs.(3)
Thus, considering that it is crucial to identify and use appropriate performance metrics that guide health managers’ decision-making, as well as the importance of developing research focused on the needs and perspectives of these professionals in the selection of KPIs,(2,3,6,9) this study proposed to identify environmental performance indicators that can collaborate with more sustainable health services, and validate them with experts.
To develop environmental performance indicators and validate them with experts in the areas of health service management and/or sustainability.
Methods
This is an applied methodological study with a mixed approach, developed in two main stages: development and validation of environmental performance indicators. The research is classified as applied because it sought to develop a set of indicators with potential for practical use in the management of health services, contributing directly to the improvement of processes and decision-making in the area.
In the first stage, the indicators to be evaluated were developed, based on the 10 objectives of the Global Agenda for Green and Healthy Hospitals of the International Organization Health Without Harm, which were subsequently adapted to the context of Brazilian health services. This refers to a document that includes a comprehensive environmental health agenda for hospitals and health systems around the world, so that they can operate in a more sustainable and climate-smart manner.(10)
The second stage consisted of validating the identified indicators using the Delphi Technique, applied to a panel of experts selected according to previously defined criteria. This stage included: selecting participants, formulating questions for the questionnaire, analyzing data, compiling responses and sending them back to the experts for consensus and/or completion of their responses. The technique was carried out in two rounds, seeking consensus on the clarity, relevance and pertinence of the proposed indicators.
Data collection was carried out with 5 experts from the southeast region of Brazil and the survey was administered via a virtual environment. The first round took place between June and August 2024 and the second round between October and November 2024.
The experts were pre-selected based on their CVs available on the Lattes Platform and Linkedin. The selection criteria for the experts took into account aspects such as: professional qualifications, experience in the area of health management and/or in relation to environmentally sustainable practices in health services; and having developed studies on the subject.
The experts were invited to participate in the research through an explanatory letter sent by email. After expressed interest in participating in the research, a link was sent to a Google Docs document containing the data collection instrument divided into three sections: Section 1 - Free and Informed Consent Record; Section 2 - Expert information: with questions to characterize the professionals; and Section 3 - Identification of environmental performance indicators: with questions regarding the indicators.
The Likert Scale was used to answer the questions in section 3 in order to standardize and quantify the evaluation. This psychometric scale was divided into five different levels of importance. At the end of each type of resource questioned, there was an open question so that the experts could write comments or suggestions.
For statistical analysis, numbers associated with the different levels of importance were assigned: 1 – not important, 2 – sometimes important, 3 – moderate, 4 – important, 5 – very important.
Next, with the help of a statistician, the Content Validity Index (CVI) was applied to measure the percentage of experts who agreed with the indicators presented. For the degree of validity, the cut-off criterion considered for response was 80% acceptance, that is, indicators that 80% or more of the experts evaluated as important indicators for decision-making regarding the practice of sustainable actions in health services were considered. After the first statistical analysis, the material was made available to the experts in the second round, excluding the indicators accepted according to the cut-off criteria, and forwarding those that did not achieve consensus for new notes or confirmation of the previously given response.
Finally, Cronbach’s Alpha coefficient was also calculated at the end of each round, as a way of estimating the reliability of the results.
This study was approved by the Research Ethics Committee of the Universidade Federal de São Paulo – UNIFESP, through opinion number 6,469,440 dated October 29, 2023 (Certificado de Apresentação para Apreciação Ética: 70315423.0.0000.5505). All participants involved agreed and signed the Free and Informed Consent Registry for Research in a Virtual Environment.
Results
Forty-four environmental performance indicators were initially developed based on the 10 objectives of the Global Agenda for Green and Healthy Hospitals. This document includes the areas: leadership, chemicals, waste, energy, water, transportation, food, pharmaceuticals, buildings, and purchasing.(10) So, we adapted and grouped the indicators into 8 categories: 1. energy resource management; 2. water resource management; 3. supply chain management; 4. management of the transportation system; 5. food management; 6. waste management; 7. facilities and infrastructure; and 8. greenhouse gas emissions.
Then, the questionnaire containing 44 indicators grouped into 8 categories was analyzed by the panel of experts. At the end of each category, there was an open-ended question allowing the expert to add comments or suggestions regarding the indicators related to the type of resource addressed. Of the 7 professionals invited, 5 (71.4%) responded that they were interested in participating in the research, and all of these answered the questionnaires in both rounds.
Regarding the characterization of the professionals, of the five who composed the expert panel, 2 (40%) were male and 3 (60%) were female, from different professions: 2 nurses, 1 physician, 1 environmental engineer, and 1 architect. All (100%) held positions related to health services at the time of the survey. Four (80%) had stricto sensu postgraduate degrees (master’s, doctorate, and post doctorate), and only one (20%) had a lato sensu postgraduate degree (specialization).
About the experience of the specialists, 3 (60%) of the specialists have experience in health services management, with 2 (40%) having more than 10 years of experience. Of the five, 4 (80%) have experience in environmentally sustainable practices, and of these 3 (60%) also have published studies and research on the subject. Thus, all (100%) have experience in health management or experience in environmentally sustainable practices in health services.
After the participants completed the questionnaire, the responses were analyzed quantitatively. Indicators with a CVI score of 1 or 0.8 were considered important. Indicators with a CVI score between 0.6 and 0.4 were sent again in the second round for reassessment, as there was no agreement. Indicators with a CVI score of 0.2 were excluded due to the agreement among the experts that they were not important (Table 1). Cronbach’s alpha was = 0.749.
By consensus, of the total of 44 indicators, 30 (68.2%) were considered important in the opinion of the experts and 4 (9.1%) were not considered important. These indicators were excluded from the first round. The other 10 indicators (22.73%) did not reach consensus, as shown in table 1.
The 10 (22,83%) indicators that did not reach consensus in the first round were included in the second round questionnaire along with 10 other indicators suggested by participants when answering the open questions, which included: energy consumption per employee - FTE (full time equivalent); energy consumption per service (for units that do not have hospitalization); energy consumption per m2 of built area; water consumption per working time of each employee - FTE; average number of kilometers traveled by employees for commuting (home-work and work-home); amount of animal proteins consumed per patient/day; amount of animal proteins consumed per employee/day; total waste generated by the institution in the month; amount of batteries discarded; amount in m2 of trees and other types of vegetation in the uncovered area of the institution. In this round, adjustments were also made to the questions in order to clarify the experts’ doubts, as well as suggestions for adapting the indicators. In fact, the indicators “number of suppliers that follow the institution’s sustainability criteria” and “number of suppliers by performance category” were replaced by the indicator “total number of suppliers versus suppliers that follow the sustainability criteria” as suggested by an expert and evaluated in the second round by the others. Thus, as shown in table 2, 20 indicators were evaluated in the second round. Cronbach’s alpha in the second round of evaluation was = 0.708. Of the total, 15 had a CVI with a result equal to 1 or 0.8 and were considered more important in the opinion of the experts. The other 5 indicators with CVI below 0.6 were definitively excluded. However, after analyzing both rounds, 44 environmental indicators were defined that were considered important for decision-making regarding the practice of sustainable actions in health services, compiled in table 3.
Discussion
The characterization of the professionals shows that, although it was an apparently small group, it was a heterogeneous group, with participation from both genders and different professions. Representativeness in the Delphi Technique is assessed based on the quality of the panel of experts and not on their numbers. Although the literature presents controversy regarding this, some authors recommend a minimum of five and a maximum of ten people participating in this process for content validation.(11)Heterogeneous groups lead to better performances and higher quality responses due to a wider range of perspectives.(12)The fact that the experts had high qualifications, all of them postgraduate degrees, allowed for a greater understanding of the content, although adjustments to the writing and comprehension of the text were requested and adjusted for the second round.
The experts’ expertise is also evidenced through their experience in health service management or environmentally sustainable practices, some with scientific publications on the subject, which is consistent with the guidelines that members be chosen for their work in the appropriate area and for their credibility.(13)
Despite the different characteristics and experiences of the panel of experts, even in the first round there was consensus regarding the degree of importance of most indicators, demonstrating agreement regarding their opinions regarding the KPIs.
The KPIs related to energy resources initially presented in the questionnaire were all included after the two rounds, with three more indicators being added by the experts. This corroborates other research that indicated that KPIs on energy use and GHG emissions are of great importance to stakeholders.(3)Energy consumption in hospitals is high, but by analyzing the causes of energy consumption in hospital buildings, identifying the main consuming structures, modernizing and modifying existing systems, and designing the photovoltaic power generation system in combinations with local climate characteristics, energy savings and emission reductions can be achieved simultaneously.(14)
Regarding water resources, the experts considered most of the suggested KPIs to be important. We can and should ensure sustainable water management in healthcare facilities by using this natural resource efficiently.(15)However, the indicator related to water consumption for cleaning transport vehicles was not considered important for decision-making by the experts. Similarly, indicators related to the transportation system, such as: number of kilometers traveled externally for patient transportation; average number of people per trip and number of deliveries made per trip, were excluded. However, they considered the estimate of greenhouse gas (GHG) emissions due to patient transportation and deliveries made by the institution to be important. One of the experts also suggested adding the indicator “average number of kilometers traveled by employees for commuting (home-work and work-home)”, which was considered important in the second round by the other professionals. In another survey, conducted with healthcare leaders in environmental management, the interviewees also considered that individuals can impact transportation emissions through their travel choices.(16)
The KPIs related to the supply chain that are considered important are related not only to the acquisition of materials, but also to their reuse, in addition to the identification of suppliers that follow the sustainability criteria, which must be established by each institution. Although other studies have identified several performance indicators related to the hospital supply chain, these are mainly related to financial, managerial and clinical aspects.(2) However, within the systemic concept that we must all be committed to the goals of sustainable development,(17) environmental aspects must be considered when establishing the KPIs for this very important area within hospitals. One of the greatest areas suitable for improving environmentally responsible health does not reside in the direct provision of care, but in the supply chains, on which we depend to bring the necessary materials and equipment to the hands of health professionals. Selecting the most sustainable product requires the evaluation of aspects of manufacturing and production that are often not presented clearly.(18)
The quality of suppliers, the amount of waste and reuse are not only important in terms of materials, but also in terms of the food purchased and provided by health services. Food waste in hospitals has traditionally been much higher than in other areas of the food supply. The cause of this situation has many negative impacts on health, the economy, society and the environment.(19)Other authors confirm the importance of measuring and monitoring food waste as a way to raise awareness among those involved and promote actions to reduce waste.(20)In addition, there is a wide variety of strategies related to food waste, the most effective of which are those that provide choice to the patient and reduce the time between ordering and eating.(20)To achieve improved nutrition and promote sustainable agriculture, we can also select the origin of food purchased from health institutions, giving preference to local food producers.(15,20)
Quantifying the different types of waste discarded by health services has also proven to be important in the opinion of experts, in addition to identifying the quantity of batteries consumed. Many of the sustainable actions carried out in health services will directly impact the total amount of waste generated, therefore, it will reflect not only the specific actions for waste disposal, but also other actions related to the reduction, recycling and reuse stages, as evidenced in another study.(21)Therefore, KPIs related to waste can be used as an important indicator for analyzing different practical interventions.
Regarding physical facilities, the same study also highlighted the importance of the hospital having low-energy light bulbs, as well as taps with flow regulators, automatic closing or with a sensor to reduce water consumption.(21)Quantifying this environmentally friendly equipment, as well as quantifying the number of trees and other types of vegetation in the institution’s open area by square meters (m2), were also considered important KPIs, in the experts’ view.
Although many of the actions related to the aforementioned KPIs directly impact GHG emissions, other indicators related to this topic were also presented. The carbon footprint per surgery has been a concern addressed in several other studies(22-25)and, according to the results of this study, it is also a concern among experts. Operating rooms have a significant environmental impact due to the high consumption of materials, generation of waste, and anesthetic agents. Quantifying the consumption of gauzes and characterizing them by type and filminess is important, since gases such as desflurane and N2O are anesthetic gases that must be avoided due to their high global warming potential (GWP) and their impact on the ozone layer.(22)
Although the KPI related to the amount of reused/repurposed construction material was excluded, the KPI on the estimate of carbon emissions by items under construction and renovation in the institution was considered important. In the hospital area, organizations that respect the environment, starting with their construction, have also been recognized for their commitment to the environment and society.(4)Thinking about environmental aspects, from the moment a construction or renovation project is drawn up, will help reduce its impact on nature, as well as help reduce costs for the institution.
Finally, the Cronbach’s alpha result, which presented an average of 0.7 in both rounds, demonstrated moderate consistency of the indicators. This is probably due to the unprecedented nature of this type of evaluation, with a need for practical application to improve the definition of indicators in the daily life of organizations. On the other hand, these different indicators can indicate the path, regarding sustainable actions in health services.
Conclusion
Forty-four environmental performance indicators have been developed and validated by experts as important to assist professionals in making decisions about sustainable practices in health services. Having metrics that allow monitoring not only the results and achievement of institutional objectives, but also carbon emissions, could contribute to reduce the effects of human action on the environment.
References
- 1 Victor S, Farooq A. Dashboard Visualisation for healthcare performance management: balanced scorecard method. Asia Pac J Health Manag. 2021;16(2):i625.
- 2 Fallahnezhad M, Langarizadeh M, Vahabzadeh A. Key performance indicators of hospital supply chain: a systematic review. BMC Health Serv Res. 2024;24(1):1610.
- 3 Zarzycka E, Krasodomska J. Environmental key performance indicators: the role of regulations and stakeholder influence. Environ Syst Decis. 2021;41(4):651-66.
- 4 Furukawa PO, Cunha IC, Pedreira MD. Time to act sustainably: why can't we wait any longer? Rev Bras Enferm. 2023;76(5):e20220813.
- 5 Rodrigues FA, de Souza MA, Sousa ZS, Lacerda DS. Indicadores ambientais: instrumentos de gestão ambiental. Cuad Educ Desarollo. 2023;15(12):15234-56.
- 6 Galvão DM, Cezar-Vaz MR, Xavier DM, Penha JG, Lourenção LG. Hospital sustainability indicators and reduction of socio-environmental impacts: a scoping review. Rev Esc Enferm USP. 2023;57:e20220364.
- 7 Li S, Xu C, Su M, Lu W, Chen Q, Huang Q, et al. Downscaling of environmental indicators: A review. Sci Total Environ. 2024;916:170251.
- 8 Navi M, Hansen A, Nitschke M, Hanson-Easey S, Pisaniello D. Developing health-related indicators of climate change: australian stakeholder perspectives. int J Environ Res Public Health. 2017;14(5):552.
- 9 Zhang L, Liu R, Jiang S, Luo G, Liu HC. Identification of key performance indicators for hospital management using an extended hesitant linguistic DEMATEL approach. Healthcare (Basel). 2019;8(1):1-21.
-
10 Health Care Without Harm. Global Green and Healthy Hospitals: a comprehensive environmental health agenda for hospitals and health systems around the world. 2011 [cited 26 Apr 2025]. Available from: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/ https://global.noharm.org/media/3888/download?inline=1
» chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/» https://global.noharm.org/media/3888/download?inline=1 - 11 Alexandre NM, Coluci MZ. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciênc Saúde Coletiva. 2011; 6(7):3061-8.
- 12 Shang Z. Use of Delphi in health sciences research: A narrative review. Medicine (Baltimore). 2023;102(7):e32829.
- 13 Powell C. The Delphi technique: myths and realities. J Adv Nurs. 2003 Feb;41(4):376-82.
- 14 Yu D, Tan X, Liu Z, Li D, Wang Z, Yan P, et al. Energy saving and carbon reduction schemes for hospital with photovoltaic power generation and system upgrading technology. Heliyon. 2023;9(11):e21447.
- 15 Furukawa PO. A enfermagem e sua atuação para o alcance dos objetivos de desenvolvimento sustentável. Enferm Foco. 2024;15:e-EDT20240001.
- 16 Schenk E, Johnson S. Nurse-sensitive environmental indicators: A qualitative study. J Nurs Manag. 2022 Nov;30(8):4378-86.
-
17 United Nations. Department of Economic and Social Affairs. Sustainable development. The 17 goals. [cited 2025 Mai 1]. Available from: https://sdgs.un.org/goals
» https://sdgs.un.org/goals - 18 Ledda V, Adisa A, Agyei F, Caton L, George C, Ghaffar A, et al. Environmentally sustainable surgical systems. BMJ Glob Health. 2024;9 Suppl 4:e015066.
- 19 Antasouras G, Vasios GK, Kontogiorgis C, Ioannou Z, Poulios E, Deligiannidou GE, et al. How to improve food waste management in hospitals through focussing on the four most common measures for reducing plate waste. Int J Health Plann Manage. 2023;38(2):296-316.
- 20 Carino S, Porter J, Malekpour S, Collins J. Environmental sustainability of hospital foodservices across the food supply chain: a systematic review. J Acad Nutr Diet. 2020;120(5):825-73.
- 21 Furukawa PO, Cunha IC, Pedreira ML, Marck PB. Environmental sustainability in medication processes performed in hospital nursing care. Acta Paul Enferm. 2016;29(3):316-24.
- 22 Gordon D. Sustainability in the Operating Room: Reducing Our Impact on the Planet. Anesthesiol Clin. 2020;38(3):679-92.
- 23 Petit HJ, Sullivan GA, Hughes IM, Pittman KL, Myers JA, Cocoma SM, et al. Exploring barriers and facilitators to reducing the environmental impact of the operating room. J Surg Res. 2023;292:197-205.
- 24 Wu S, Cerceo E. Sustainability initiatives in the operating room. Jt Comm J Qual Patient Saf. 2021;47(10):663-72.
- 25 Gorgun E, Dehipawala S, O'Hara M, Naoumtchik E, Gangoli G, Ricketts C, et al. Environmental sustainability initiatives in the operating room: a scoping review. Ann Surg Open. 2024;5(3):e451.
