ABSTRACT
Cryptococcal infection is a frequent cause of mortality in Brazilian HIV-infected patients. The present study aimed to evaluate the cost-effectiveness and budgetary impact of four cryptococcosis screening strategies in HIV-infected patients with CD4+ ≤ 200 cells/mm3, in Brazil. A Monte Carlo microsimulation was used to compare the following diagnostic tests: a systematic serum cryptococcal antigen (CRAG) screening with latex agglutination (CRAG-LA), a lateral flow immunochromatographic test (CRAG-LFA), India ink staining and no intervention. The rationale was that of the Unified Health System (SUS), and the time horizon was of one year for the intervention and of five years for the budgetary impact analysis (BIA). The effectiveness outcomes were years of life and years of life adjusted for quality (QALY). The cost-effectiveness analysis showed that the two cryptococcal antigen tests were cost-effective, presenting with superior results in comparison with India ink and no screening. CRAG-LFA, compared to CRAG-LA, has an incremental cost of US$0.25 and an incremental cost-effectiveness ratio of US$73.36 (considering the US dollar equal to 5 reais, the Brazilian current money). The probabilistic sensitivity analysis between CRAG-LFA and CRAG-LA, despite showing a high agreement between the two tests, indicated the superiority of CRAG-LFA. The BIA estimated that the incorporation of CRAG-LFA would have an additional cost of approximately U$S 10.4 million dollars in five years. These findings suggest that, for the group of studied patients, the adoption of CRAG-LFA and CRAG-LA are cost-effective, while the India ink test and no intervention are less effective strategies. The BIA showed that using the CRAG-LFA test for people living with HIV (PLHIV) with CD4+ ≤ 200 cells/mm3 could reduce costs for the Brazilian Unified Health System (SUS).
Cost-benefit analysis; Diagnostic test approval; Cryptococcus; HIV infections
INTRODUCTION
Cryptococcosis is a systemic mycosis caused by fungi of the genus Cryptococcus. The infection typically affects the central nervous system (CNS) and the respiratory tract, commonly presenting as cryptococcal meningitis (CM). HIV infection with a CD4 count below 200 cells/mm33. Pappalardo MC, Melhem MS. Cryptococcosis: a review of the Brazilian experience for the disease. Rev Inst Med Trop Sao Paulo. 2003;45:299-305. and the presence of the acquired immunodeficiency syndrome (AIDS) are predisposing conditions for cryptococcosis11. World Health Organization. Guidelines for the diagnosis, prevention and management of cryptococcal disease in HIV-infected adults, adolescents and children: supplement to the 2016 consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. Geneva: WHO; 2018. [cited 2021 Jun 22]. Available from: http://apps.who.int/iris/bitstream/handle/10665/260399/9789241550277-eng.pdf;jsessionid=3DD47D6B4F06AFA32280E2048900A316?sequence=1
http://apps.who.int/iris/bitstream/handl...
.
CM is associated with about 15% of all HIV deaths. In 2014, a total of 223,100 CM cases led to 181,100 deaths, 75% of which in sub-Saharan Africa, followed by Southeast Asia and Latin America22. Rajasingham R, Smith RM, Park BJ, Jarvis JN, Govender NP, Chiller TM, et al. Global burden of disease of HIV-associated cryptococcal meningitis: an updated analysis. Lancet Infect Dis. 2017;17:873-81.. In Brazil, the cryptococcosis mortality rate is between 45% to 65% regardless of HIV risk factors and the primary form of the disease33. Pappalardo MC, Melhem MS. Cryptococcosis: a review of the Brazilian experience for the disease. Rev Inst Med Trop Sao Paulo. 2003;45:299-305.. Cryptococcosis occurs as the first opportunistic manifestation in about 4.4% of AIDS cases in Brazil33. Pappalardo MC, Melhem MS. Cryptococcosis: a review of the Brazilian experience for the disease. Rev Inst Med Trop Sao Paulo. 2003;45:299-305.,44. Consenso em criptococose: 2008. Rev Soc Bras Med Trop. 2008;41:524-44.. In addition, patients recovering from cryptococcal meningitis may experience long-term neurological and sensory impairment (≥ 3 months after the diagnosis), resulting in disability and poor quality of life55. Vidal JE, Toniolo C, Paulino A, Colombo A, dos Anjos Martins M, da Silva Meira C, et al. Asymptomatic cryptococcal antigen prevalence detected by lateral flow assay in hospitalised HIV-infected patients in São Paulo, Brazil. Trop Med Int Heal. 2016;21:1539-44.,66. Merry M, Boulware DR. Cryptococcal meningitis treatment strategies affected by the explosive cost of flucytosine in the United States: a cost-effectiveness analysis. Clin Infect Dis. 2016;62:1564-8. a tertiary referral hospital to HIV-infected patients serving the São Paulo State, Brazil. All patients were >18 years old without prior cryptococcal meningitis, without clinical suspicion of cryptococcal meningitis, regardless of antiretroviral (ART.
The Unified Health System (SUS) provides different methods for diagnosing Cryptococcosis. The conventional one is the direct examination of slides stained with India ink and the sample’s culture on Sabouraud dextrose agar (SDA)44. Consenso em criptococose: 2008. Rev Soc Bras Med Trop. 2008;41:524-44.. In the case of Cryptococcosis, the India ink method refers to the direct microscopy of the cerebrospinal fluid (CSF), secretions or exudates, subcutaneous aspirate, pus, urine, serum, plasma or blood to visualize yeasts44. Consenso em criptococose: 2008. Rev Soc Bras Med Trop. 2008;41:524-44.. In contrast, the SDA culture consists of yeast culture for seven days, between 25 °C to 37 °C4. Serological techniques can be used to detect antigens (CRAG) or search for antibodies. The latex agglutination test (CRAG-LA) is one of the most frequently used due to its high accuracy44. Consenso em criptococose: 2008. Rev Soc Bras Med Trop. 2008;41:524-44.,77. Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Vigilância, Prevenção e Controle das Infecções Sexualmente Transmissíveis, do HIV/Aids e das Hepatites Virais. Protocolo clínico e diretrizes terapêuticas para manejo da infecção pelo HIV em adultos. Brasília: Ministério da Saúde; 2018. [cited 2021 Jun 22]. Available from: http://www.aids.gov.br/pt-br/pub/2013/protocolo-clinico-e-diretrizes-terapeuticas-para-manejo-da-infeccao-pelo-hiv-em-adultos
http://www.aids.gov.br/pt-br/pub/2013/pr...
along with the lateral flow immunochromatographic test (LFA)88. Vidal JE, Boulware DR. Lateral flow assay for cryptococcal antigen: an important advance to improve the continuum of HIV care and reduce cryptococcal meningitis-related mortality. Rev Inst Med Trop Sao Paulo. 2015;57 Suppl 19:38-45. the cryptococcal antigen lateral flow assay (CrAg LFA. The CRAG-LFA is a rapid diagnostic test that provides a definitive result in ≤ 10 min. with no need of specialized physical structure. The technique is stable at room temperature and easy to use and interpret results88. Vidal JE, Boulware DR. Lateral flow assay for cryptococcal antigen: an important advance to improve the continuum of HIV care and reduce cryptococcal meningitis-related mortality. Rev Inst Med Trop Sao Paulo. 2015;57 Suppl 19:38-45. the cryptococcal antigen lateral flow assay (CrAg LFA.
The Brazilian Ministry of Health (MH) Clinical Protocol and Therapeutic Guideline (CPTG) for the treatment of adults with HIV recommends that people living with HIV (PLHIV) should to be investigated for fungal etiology in all pneumonia cases in addition to the routinely performed exams77. Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Vigilância, Prevenção e Controle das Infecções Sexualmente Transmissíveis, do HIV/Aids e das Hepatites Virais. Protocolo clínico e diretrizes terapêuticas para manejo da infecção pelo HIV em adultos. Brasília: Ministério da Saúde; 2018. [cited 2021 Jun 22]. Available from: http://www.aids.gov.br/pt-br/pub/2013/protocolo-clinico-e-diretrizes-terapeuticas-para-manejo-da-infeccao-pelo-hiv-em-adultos
http://www.aids.gov.br/pt-br/pub/2013/pr...
. Those who are CRAG-positive and present with CD4 ≤ 100 cells/mm3 are subjected to preemptive treatment with fluconazole and antiretroviral therapy (ART) until they reach CD4 > 200 cells/mm3. The cost-effectiveness of CRAG screening, in either asymptomatic or symptomatic CM patients, has never been evaluated in Brazil.
The primary aim of this study was to model the cost-effectiveness and budget impact of implementing CRAG screening for asymptomatic cryptococcus infections in PLHIV with CD4+ ≤ 200 cells/mm3, in Brazil.
MATERIALS AND METHODS
A Monte Carlo analytical decision model assessed the cost-effectiveness ratio and budgetary impact of the screening of PLHIV, over 18 years old, of both genders, infected with cryptococcosis, asymptomatic, with CD4 ≤ 200 cells/mm3, using the following methods as screening strategies: (a) the point-of-care diagnostic test CRAG-LFA, (b) cryptococcus research in CSF with India ink or (c) CRAG-LA in serum, using no intervention as the baseline. Each strategy has its own accuracy value and employs different sample collection methods. This type of analysis records the patient’s history and the transition probabilities depend on the variables involved, allowing great clinical complexity in the development of the model.
The perspective adopted considered the SUS as the funding agency for the services. The horizon for the cost-effectiveness analysis was one year for the intervention, evaluating its impact on the disease’s development for five years, as many patients have life expectancies almost equal to those of people without HIV99. Brasil. Ministério da Saúde. Indicadores e dados básicos do HIV/AIDS nos municípios brasileiros. [cited 2021 Jun 22]. Available from: http://indicadores.aids.gov.br/
http://indicadores.aids.gov.br/...
. A discount rate of 3% per year was applied1010. Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia Diretrizes metodológicas : diretriz de avaliação econômica. 2a ed. Brasília: Ministério da Saúde; 2014. [cited 2021 Jun 22]. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/diretrizes_metodologicas_diretriz_avaliacao_economica.pdf
https://bvsms.saude.gov.br/bvs/publicaco...
.
The primary outcomes were years of life and quality-adjusted life years (QALY)22. Rajasingham R, Smith RM, Park BJ, Jarvis JN, Govender NP, Chiller TM, et al. Global burden of disease of HIV-associated cryptococcal meningitis: an updated analysis. Lancet Infect Dis. 2017;17:873-81.,1111. Miot J, Leong T, Takuva S, Parrish A, Dawood H. Cost-effectiveness analysis of flucytosine as induction therapy in the treatment of cryptococcal meningitis in HIV-infected adults in South Africa. BMC Health Serv Res. 2020 In Press.. Estimates referring to direct medical costs included the identification, measurement and evaluation of the applied resources. A sensitivity analysis1010. Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia Diretrizes metodológicas : diretriz de avaliação econômica. 2a ed. Brasília: Ministério da Saúde; 2014. [cited 2021 Jun 22]. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/diretrizes_metodologicas_diretriz_avaliacao_economica.pdf
https://bvsms.saude.gov.br/bvs/publicaco...
assessed the uncertainties of the model.
According to the CPTG for the management of HIV infection in adults77. Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Vigilância, Prevenção e Controle das Infecções Sexualmente Transmissíveis, do HIV/Aids e das Hepatites Virais. Protocolo clínico e diretrizes terapêuticas para manejo da infecção pelo HIV em adultos. Brasília: Ministério da Saúde; 2018. [cited 2021 Jun 22]. Available from: http://www.aids.gov.br/pt-br/pub/2013/protocolo-clinico-e-diretrizes-terapeuticas-para-manejo-da-infeccao-pelo-hiv-em-adultos
http://www.aids.gov.br/pt-br/pub/2013/pr...
, the screening and preemptive treatment strategy for cryptococcosis prevent deaths in individuals with ART CD4+ ≤ 100 cells/mm3. The document highlights patients without clinical manifestations of cryptococcosis but presenting with a test showing positive detection of cryptococcal antigenemia in CSF samples obtained by lumbar puncture (LP), performed to rule out CM. Then, a preemptive treatment with 800 mg fluconazole per day for two weeks and 400 mg fluconazole per day for eight weeks is prescribed and ART will be administered after the first two weeks of antifungal treatment44. Consenso em criptococose: 2008. Rev Soc Bras Med Trop. 2008;41:524-44.,77. Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Vigilância, Prevenção e Controle das Infecções Sexualmente Transmissíveis, do HIV/Aids e das Hepatites Virais. Protocolo clínico e diretrizes terapêuticas para manejo da infecção pelo HIV em adultos. Brasília: Ministério da Saúde; 2018. [cited 2021 Jun 22]. Available from: http://www.aids.gov.br/pt-br/pub/2013/protocolo-clinico-e-diretrizes-terapeuticas-para-manejo-da-infeccao-pelo-hiv-em-adultos
http://www.aids.gov.br/pt-br/pub/2013/pr...
. All individuals with positive plasma or serum tests are investigated for CM using LP. The treatment of positive cases follow these phases: (a) induction, (b) consolidation and (c) maintenance, recommended by the Brazilian consensus and CPTG44. Consenso em criptococose: 2008. Rev Soc Bras Med Trop. 2008;41:524-44.,77. Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Vigilância, Prevenção e Controle das Infecções Sexualmente Transmissíveis, do HIV/Aids e das Hepatites Virais. Protocolo clínico e diretrizes terapêuticas para manejo da infecção pelo HIV em adultos. Brasília: Ministério da Saúde; 2018. [cited 2021 Jun 22]. Available from: http://www.aids.gov.br/pt-br/pub/2013/protocolo-clinico-e-diretrizes-terapeuticas-para-manejo-da-infeccao-pelo-hiv-em-adultos
http://www.aids.gov.br/pt-br/pub/2013/pr...
.
SUS’s reimbursement amounts for different items were used as measures of the abovementioned costs. Procedures and test values were obtained from the Ministry of Health1212. Brasil. Ministério da Saúde. Banco de preços em saúde. [cited 2021 Jun 22]. Available from: http://bps.saude.gov.br/login.jsf
http://bps.saude.gov.br/login.jsf...
13. Brasil. Ministério da Saúde. SIGTAP: Sistema de Gerenciamento da Tabela de Procedimentos, Medicamentos e OPM do SUS. [cited 2021 Jun 22]. Available from: http://sigtap.datasus.gov.br/tabela-unificada/app/sec/inicio.jsp
http://sigtap.datasus.gov.br/tabela-unif...
-1414. Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia, Inovação e Insumos Estratégicos em Saúde. Coordenação de Monitoramento e Avaliação de Tecnologias em Saúde. Teste diagnóstico, point of care, de Cryptococcal Antigen Lateral Flow Assay (CRAG-LFA) para detecção de infecção por Cryptococcus e diagnóstico de meningite criptocócica em pessoas vivendo com o vírus da imunodeficiência humana (PVHIV). Brasília: Ministério da Saúde; 2021. [cited 2021 Jun 22]. Available from: http://www.conitec.gov.br/images/Relatorios/2021/20210611_Relatorio_615_CragLFA_meningite-criptococica_Final.pdf
http://www.conitec.gov.br/images/Relator...
and, when not available, from the Brazilian Hierarchical Classification of Medical Procedures of the Brazilian Medical Association (BMA)1515. Associação Médica Brasileira. CBHPM-2016 : Classificação Brasileira Hierarquizada de Procedimentos Médicos. Barueri: Manole; São Paulo: Associação Médica Brasileira; 2016. [cited 2021 Jun 22]. Available from: https://www.cbo.net.br/novo/publicacoes/CBHPM_2016.pdf
https://www.cbo.net.br/novo/publicacoes/...
in addition to several consultations of studies by Lofgren et al.1616. Lofgren SM, Nalintya E, Meya DB, Boulware DR, Rajasingham R. A qualitative evaluation of an implementation study for cryptococcal antigen screening and treatment in Uganda. Medicine (Baltimore). 2018;97:e11722. implementation studies and evaluations of how to integrate CrAg screening programs into existing HIV care infrastructure are lacking. During a CrAg screening program in Kampala, Uganda, we interviewed 15 health care workers (2 coordinating research nurses and 13 clinic personnel and Rajasinghan et al.1717. Rajasingham R, Meya DB, Greene GS, Jordan A, Nakawuka M, Chiller TM, et al. Evaluation of a national cryptococcal antigen screening program for HIV-infected patients in Uganda: a cost-effectiveness modeling analysis. PLoS One. 2019;14:e0210105. and CrAg positivity is an independent predictor of meningitis and death. CrAg screening for patients with advanced HIV and preemptive treatment is recommended by the World Health Organization, though implementation remains limited. Our objective was to evaluate costs and mortality reduction (lives saved Table 1 shows the values adopted for the procedures and treatments.
Assumptions
The parameter assumptions was based on prospective CRAG studies and Brazilian consensus44. Consenso em criptococose: 2008. Rev Soc Bras Med Trop. 2008;41:524-44.,1717. Rajasingham R, Meya DB, Greene GS, Jordan A, Nakawuka M, Chiller TM, et al. Evaluation of a national cryptococcal antigen screening program for HIV-infected patients in Uganda: a cost-effectiveness modeling analysis. PLoS One. 2019;14:e0210105.
18. Ramachandran A, Manabe Y, Rajasingham R, Shah M. Cost-effectiveness of CRAG-LFA screening for cryptococcal meningitis among people living with HIV in Uganda. BMC Infect Dis. 2017;17:225.
19. Meya DB, Manabe YC, Castelnuovo B, Cook BA, Elbireer AM, Kambugu A, et al. Cost‐effectiveness of serum cryptococcal antigen screening to prevent deaths among HIV‐infected persons with a CD4 + cell count ≤100 Cells/μL who start HIV therapy in resource‐limited settings. Clin Infect Dis. 2010;51:448-55.
20. Longley N, Jarvis JN, Meintjes G, Boulle A, Cross A, Kelly N, et al. Cryptococcal antigen screening in patients initiating ART in South Africa: a prospective cohort study. Clin Infect Dis. 2016;62:581-7.
21. Temfack E, Rim JJ, Spijker R, Loyse A, Chiller T, Pappas PG, et al. Cryptococcal antigen in serum and cerebrospinal fluid for detecting cryptococcal meningitis in adults living with human immunodeficiency virus: systematic review and meta-analysis of diagnostic test accuracy studies. Clin Infect Dis. 2021;72:1268-78.
22. Nalintya E, Kiggundu R, Meya D. Evolution of cryptococcal antigen testing: what is new? Curr Fungal Infect Rep. 2016;10:62-7.-2323. Boulware DR, Rolfes MA, Rajasingham R, von Hohenberg M, Qin Z, Taseera K, et al. Multisite validation of cryptococcal antigen lateral flow assay and quantification by laser thermal contrast. Emerg Infect Dis. 2014;20:45-53. with a cohort of HIV-infected patients with CD4 < 100 cells/uL. Primary outcomes were expected costs, DALYs, and incremental cost-effectiveness ratios (ICERs. CRAG-positive individuals (CRAG+) may be asymptomatic and eligible for the preventive treatment with fluconazole in the model. International studies were used1111. Miot J, Leong T, Takuva S, Parrish A, Dawood H. Cost-effectiveness analysis of flucytosine as induction therapy in the treatment of cryptococcal meningitis in HIV-infected adults in South Africa. BMC Health Serv Res. 2020 In Press.,1717. Rajasingham R, Meya DB, Greene GS, Jordan A, Nakawuka M, Chiller TM, et al. Evaluation of a national cryptococcal antigen screening program for HIV-infected patients in Uganda: a cost-effectiveness modeling analysis. PLoS One. 2019;14:e0210105.
18. Ramachandran A, Manabe Y, Rajasingham R, Shah M. Cost-effectiveness of CRAG-LFA screening for cryptococcal meningitis among people living with HIV in Uganda. BMC Infect Dis. 2017;17:225.-1919. Meya DB, Manabe YC, Castelnuovo B, Cook BA, Elbireer AM, Kambugu A, et al. Cost‐effectiveness of serum cryptococcal antigen screening to prevent deaths among HIV‐infected persons with a CD4 + cell count ≤100 Cells/μL who start HIV therapy in resource‐limited settings. Clin Infect Dis. 2010;51:448-55.,2424. Morawski BM, Boulware DR, Nalintya E, Kiragga A, Kazooza F, Rajasingham R, et al. Pre-ART cryptococcal antigen titer associated with preemptive fluconazole failure. In: Conference on Retroviruses and Opportunistic Infections; 2016 Feb 22-25; Boston, Massachusetts. [cited 2021 Jun 22]. Available from: https://www.croiconference.org/abstract/pre-art-cryptococcal-antigen-titer-associated-preemptive-fluconazole-failure/
https://www.croiconference.org/abstract/...
the prevalence of cryptococcal antigenemia (CrAg+ in the absence of studies on Brazilian population’s quality of life for the outcomes described.
The model did not incorporate the ART treatment77. Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Vigilância, Prevenção e Controle das Infecções Sexualmente Transmissíveis, do HIV/Aids e das Hepatites Virais. Protocolo clínico e diretrizes terapêuticas para manejo da infecção pelo HIV em adultos. Brasília: Ministério da Saúde; 2018. [cited 2021 Jun 22]. Available from: http://www.aids.gov.br/pt-br/pub/2013/protocolo-clinico-e-diretrizes-terapeuticas-para-manejo-da-infeccao-pelo-hiv-em-adultos
http://www.aids.gov.br/pt-br/pub/2013/pr...
after using antifungals, the adverse effects resulting from amphotericin, and intracranial hypertension (CH) as a complication of CM. The use of amphotericin B deoxycholate (1 mg/kg/week) and other drugs to replace fluconazole was not considered.
Economic model
Figure 1 presents a schematic representation of the transition states for CM screening and treatment, with two residual states where the patient dies or has predominantly negative CM symptoms. Data from the Vidal et al.55. Vidal JE, Toniolo C, Paulino A, Colombo A, dos Anjos Martins M, da Silva Meira C, et al. Asymptomatic cryptococcal antigen prevalence detected by lateral flow assay in hospitalised HIV-infected patients in São Paulo, Brazil. Trop Med Int Heal. 2016;21:1539-44. a tertiary referral hospital to HIV-infected patients serving the São Paulo State, Brazil. All patients were >18 years old without prior cryptococcal meningitis, without clinical suspicion of cryptococcal meningitis, regardless of antiretroviral (ART were used to calibrate the model.
Schematic model of transition states for the screening of cryptococcal meningitis (CM) in asymptomatic PLHIV with ≤ 200 cells/mm3. CM = Cryptococcal meningitis; CRAG = Cryptococcal antigen.
As CM and mortality progression depends on CD4 levels, the target population had two PLHIV subgroups: CD4 ≤ 100 cells/mm3 and CD4 ≤ 200 cells/mm3. For each subgroup, after screening, those who were CRAG+ undergo preemptive treatment. After this stage, these individuals may or may not progress to CM. Those who developed CM undergo treatment consisting of induction, consolidation and maintenance. After the end of the treatment, those who did not die progress to a state without CM.
All CRAG-negative individuals, considered without CM, could develop meningitis depending on their true or false negative condition. Those who have undergone preemptive treatment could also progress to a CM status at any time, just as anyone who had finished the in-hospital therapy and may have a relapsed or contracted CM again.
Sensitivity analysis
Deterministic and probabilistic sensitivities analyzed uncertainties of the model. The discount rate varied from 0% to 5%, and medical costs were estimated by values found in public purchasing bases.
Patients’ utilities and average medical costs changed in a directly proportional manner. Medical costs are related to SUS’s services and showed similar results among patients, regardless of the therapeutic alternatives. A scatterplot was constructed in the probabilistic sensitivity analysis indicating the option that was most likely to be cost-effective.
Budget impact
The budgetary impact analysis employed the demand method2525. Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia. Diretrizes metodológicas: análise de impacto orçamentário : manual para o sistema de saúde do Brasil. Brasília: Ministério da Saúde; 2012. [cited 2021 Jun 22]. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/diretrizes_metodologicas_analise_impacto.pdf
https://bvsms.saude.gov.br/bvs/publicaco...
. The use of the CRAG-LFA diagnostic test in SUS was simulated for five years to screen for cryptococcus infection in PLHIV with CD4+ ≤ 200 cells/mm3. The BIA adopted the SUS perspective2525. Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia. Diretrizes metodológicas: análise de impacto orçamentário : manual para o sistema de saúde do Brasil. Brasília: Ministério da Saúde; 2012. [cited 2021 Jun 22]. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/diretrizes_metodologicas_analise_impacto.pdf
https://bvsms.saude.gov.br/bvs/publicaco...
. The price of the implemented technology was U$S 6 per test. The costs to assess the budgetary impact were as follows: screening –U$S 6, preemptive treatment –U$S 277.8, and meningitis treatment –U$S 2,644.60 dollars. Table 1 shows the costs of the screening and treatments.
In 2021, the Brazilian population was 212,601,219 inhabitants2626. Instituto Brasileiro de Geografia e Estatística. Projeção da população do Brasil e das unidades da federação. [cited 2021 Jun 22]. Available from: https://www.ibge.gov.br/apps/populacao/projecao/index.html?utm_source=portal&utm_medium=popclock&utm_campaign=novo_popclock
https://www.ibge.gov.br/apps/populacao/p...
. The HIV cases considered were the mean number of cases in the period from 2018 to 2020 reported to the Notifiable Diseases Information System (SINAN), declared in the Mortality Information System (SIM), and registered in the Control System for Laboratory Examinations of the National Network of CD4+/CD8+ Lymphocyte Count and HIV Viral Load (SISCEL)/Logistics Control System for Medicines (SICLOM)99. Brasil. Ministério da Saúde. Indicadores e dados básicos do HIV/AIDS nos municípios brasileiros. [cited 2021 Jun 22]. Available from: http://indicadores.aids.gov.br/
http://indicadores.aids.gov.br/...
.
RESULTS
For the effectiveness measured in years of life, the microsimulation estimated an incremental cost-effectiveness ratio (ICER) that is shown in Table 2.
The cost-effectiveness analysis showed that the two tests, CRAG-LA and CRAG-LFA, are cost-effective, a superior performance in comparison with the India ink and the no evaluation scenario.
As the CRAG-LA and CRAG-LFA tests proved to be cost-effective, a deterministic univariate sensitivity analysis evaluated the accuracy of each of the tests, their sensitivities and specificities (Table 3). The results for sensitivity showed that both tests are no longer cost-effective for values below 0.975. For specificity, this response occurs for CRAG-LFA above 0.9 and for CRAG-LA with values around 0.99. The price of CRAG-LA was changed to assess its impact in the result. For the same price as CRAG-LFA, of U$S 6 per test, the alternative to use CRAG-LA was superior.
The probabilistic sensitivity analysis between CRAG-LFA and CRAG-LA, although pointing to a high agreement between the two tests, indicated the superiority of CRAG-LFA (Figure 2). In approximately 38% of the cases, this test was superior to the CRAG-LA.
Budget impact
We would have a total cost for a cohort of 45,000 patients with the hypothesis of all of them treated for meningitis of approximately U$S 34.8 million dollars for five years without screening. Using the measured demand method and a five-year time horizon, an assumed initial market share of 20% for CRAG-LFA, with annual increments of the same amount, reaching 100% in the fifth year, the estimated budget impact resulting from the incorporation of the test would have an additional cost of approximately U$S 10.5 million dollars. For comparison, the budgetary impact of 100% adoption of screening for asymptomatic PLHIV CD4 ≤ 200 cells/mm3 is a five-year cost savings of U$S 11 million dollars in this context.
DISCUSSION
The diagnosis and management of cryptococcosis and its outcomes are structural problems in Brazil11. World Health Organization. Guidelines for the diagnosis, prevention and management of cryptococcal disease in HIV-infected adults, adolescents and children: supplement to the 2016 consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. Geneva: WHO; 2018. [cited 2021 Jun 22]. Available from: http://apps.who.int/iris/bitstream/handle/10665/260399/9789241550277-eng.pdf;jsessionid=3DD47D6B4F06AFA32280E2048900A316?sequence=1
http://apps.who.int/iris/bitstream/handl...
,44. Consenso em criptococose: 2008. Rev Soc Bras Med Trop. 2008;41:524-44.,55. Vidal JE, Toniolo C, Paulino A, Colombo A, dos Anjos Martins M, da Silva Meira C, et al. Asymptomatic cryptococcal antigen prevalence detected by lateral flow assay in hospitalised HIV-infected patients in São Paulo, Brazil. Trop Med Int Heal. 2016;21:1539-44. que fazem parte do complexo C. neoformans. Com o aparecimento da AIDS, houve expressivo aumento na incidência da criptococose. Posteriormente, o uso de antifúngicos profiláticos e a introdução da HAART levaram à redução desta incidência. Objetivamos determinar o perfil epidemiológico da meningite criptocócica no Estado do Rio de Janeiro no período de 1994 a 2004, com base nos dados da Assessoria de Meningite, setor do Centro de Vigilância Epidemiológica da SSE-RJ e, também, avaliar em que medida o perfil epidemiológico disponível no sistema nacional (SINAN. Our study suggests that, for PLHIV with CD4 ≤ 200 cells/mm3, the incorporation of CRAG-LFA or CRAG-LA in the patients’ screening would be advantageous compared to no screening and the India ink test. Our model projects a reduction in mortality due to CM in PLHIV CRAG+ CD4 ≤ 200 cells/mm3 by 15%, while costs for 1,000 patients would decrease by US$ 11 million dollars in five years compared to no screening.
Ramachandran et al.1818. Ramachandran A, Manabe Y, Rajasingham R, Shah M. Cost-effectiveness of CRAG-LFA screening for cryptococcal meningitis among people living with HIV in Uganda. BMC Infect Dis. 2017;17:225. conducted a study similar to ours in Uganda, Africa. The authors assessed the costs and cost-effectiveness of implementing a CRAG screening for PLHIV in comparison with the usual practice. The CRAG screening was considered highly cost-effective and was associated with an ICER of US$ 6.14 per avoided DALY compared to no screening. The implementation cost of CRAG screening was of an additional US$1.52 per person and resulted in a 40% relative reduction in disease-associated mortality. The probabilistic sensitivity analysis showed that the CRAG screening was cost-effective in 100% of the scenarios. A secondary analysis projected a total cost of US$651.454 for the 100% implementation of the screening in the country, preventing 1,228 deaths compared to no screening.
Another study carried out in Cambodia2727. Micol R, Tajahmady A, Lortholary O, Balkan S, Quillet C, Dousset JP, et al. Cost-effectiveness of primary prophylaxis of AIDS associated cryptococcosis in Cambodia. PLoS One. 2010;5:e13856. one time systematic serum cryptococcal antigen (CRAG evaluated the cost-effectiveness of three alternative strategies for preventing cryptococcal infection in HIV-infected patients. Opportunistic cryptococcal infection is endemic in Cambodia and corresponds to a significant public health burden in PLHIV there and in Brazil. Over a one-year horizon, the systematic CRAG screening (for targeted treatment of positive cases) in patients with CD4 ≤ 100 cells/mm3 is more cost-effective for preventing cryptococcosis than the systematic primary prophylaxis strategy.
Tenforde et al.2828. Tenforde MW, Muthoga C, Callaghan A, Ponatshego P, Ngidi J, Mine M, et al. Cost-effectiveness of reflex laboratory-based cryptococcal antigen screening for the prevention and treatment of cryptococcal meningitis in Botswana. Wellcome Open Res. 2020;4:144. made a cost-effective model of CRAG screening for CRAG+, treatment-naïve patients with CD4 ≤ 100 cells/mm3 and CRAG+ CD4 ≤ 100 cells/mm3 with previous experience of ART in Botswana. The authors estimated that of 650,000 samples submitted to the CD4 test annually, 16,364 would have CD4 ≤ 100 cells/mm3 and would undergo a CRAG test, and 70% of patients reporting previous experience with ART at the time of screening. CRAG screening and preventive treatment in CD4 treatment-naïve patients with ≤ 100 cells/mm3 prevented 20% of MC-related deaths at the cost of US$2 per avoided DALY. The expansion of the preemptive treatment, including patients with previous ART experience and CD4 ≤ 100 cells/mm33. Pappalardo MC, Melhem MS. Cryptococcosis: a review of the Brazilian experience for the disease. Rev Inst Med Trop Sao Paulo. 2003;45:299-305., would generated 55 additional cost-avoided deaths with respect to no screening.
The results of the present study deserve to be analyzed carefully. Some limitations are inherent to the modeling process, which can have oversimplified the disease’s progression due to differences in real-world circumstances and the use of more than one treatment, disease complications and adverse drug effects. Different sources of international data were the basis for estimating the transition probability values22. Rajasingham R, Smith RM, Park BJ, Jarvis JN, Govender NP, Chiller TM, et al. Global burden of disease of HIV-associated cryptococcal meningitis: an updated analysis. Lancet Infect Dis. 2017;17:873-81.,88. Vidal JE, Boulware DR. Lateral flow assay for cryptococcal antigen: an important advance to improve the continuum of HIV care and reduce cryptococcal meningitis-related mortality. Rev Inst Med Trop Sao Paulo. 2015;57 Suppl 19:38-45.,1616. Lofgren SM, Nalintya E, Meya DB, Boulware DR, Rajasingham R. A qualitative evaluation of an implementation study for cryptococcal antigen screening and treatment in Uganda. Medicine (Baltimore). 2018;97:e11722.,2020. Longley N, Jarvis JN, Meintjes G, Boulle A, Cross A, Kelly N, et al. Cryptococcal antigen screening in patients initiating ART in South Africa: a prospective cohort study. Clin Infect Dis. 2016;62:581-7. and we aimed to provide an updated estimate of global incidence of HIV-associated cryptococcal disease. Methods We used 2014 Joint UN Programme on HIV and AIDS estimates of adults (aged >15 years. In the sensitivity analysis, these variables did not affect the results. We have also considered that any CM complication would lead to hospitalization or death, which may not be accurate in all cases. The model did not include effects other than meningitis caused by cryptococcal infections and the adverse effects of treatment.
Unit costs calculated from SIGTAP1313. Brasil. Ministério da Saúde. SIGTAP: Sistema de Gerenciamento da Tabela de Procedimentos, Medicamentos e OPM do SUS. [cited 2021 Jun 22]. Available from: http://sigtap.datasus.gov.br/tabela-unificada/app/sec/inicio.jsp
http://sigtap.datasus.gov.br/tabela-unif...
may be underestimated. Two of the technologies used for comparisons had no prices indicated in the BPS1212. Brasil. Ministério da Saúde. Banco de preços em saúde. [cited 2021 Jun 22]. Available from: http://bps.saude.gov.br/login.jsf
http://bps.saude.gov.br/login.jsf...
, and the 2016 AMB table was used1515. Associação Médica Brasileira. CBHPM-2016 : Classificação Brasileira Hierarquizada de Procedimentos Médicos. Barueri: Manole; São Paulo: Associação Médica Brasileira; 2016. [cited 2021 Jun 22]. Available from: https://www.cbo.net.br/novo/publicacoes/CBHPM_2016.pdf
https://www.cbo.net.br/novo/publicacoes/...
, which may be out of date. Still, as the CRAG-LFA was superior than the other two tests, these prices have little influence on the result. In this analysis, we assumed equal hospitalization days for patients who underwent screening with any of the technologies. Because of the lower accuracy of a given method, the duration of admission, risk of readmission, and risk of complications that could prolong hospitalization therefore increasing hospital care costs and duration. The cheapest combination corresponding to the use of fluconazole was adopted on the list of CMED2929. Brasil. Agência Nacional de Vigilância Sanitária. Câmara de Regulação do Mercado de Medicamentos. Preços máximos de medicamentos por princípio ativo. Brasília; 2020. [cited 2021 Jun 22]. Available from: https://www.gov.br/anvisa/pt-br/assuntos/medicamentos/cmed/precos/anos-anteriores/arquivos/4910json-file-1
https://www.gov.br/anvisa/pt-br/assuntos...
at a maximum price to the consumer instead of the government’s purchase price. Finally, the intangible costs of using each technology have not been calculated. The result of the CRAG-LFA test from blood collection can be obtained in 10 minutes. However, CRAG-LA requires a lumbar puncture and a medical follow-up.
CONCLUSION
An economic evaluation of the Monte Carlo microsimulation estimated the main costs and consequences of the cost-effectiveness analysis and budgetary impact of implementing the CRAG-LFA test for the screening of cryptococcus infection in PLHIV with CD4+ ≤ 200 cells/mm3 in comparison with the CRAG-LA, the India ink, and no screening as the reference, from the perspective of the Brazilian Unified Health System.
Given the results of this study, the CRAG-LFA and CRAG-LA tests are potentially cost-effective to track cryptococcal infections, preventing mortality related to cryptococcal meningitis in PLHIV with CD4 ≤ 200 cells/mm3. India ink and no intervention were inferior strategies. According to the results of the model and the sensitivity analysis, CRAG screening for PLWH with CD4+ ≤ 200 cells/mm3 represents an excellent opportunity to save money with the potential to prevent CM and reduce the corresponding mortality in Brazil. The BIA showed that for a cohort of 45,000 patients without screening, we would have a total cost of approximately U$S 34.8 million dollars in five years. The budgetary impact of the 100% adoption of screening in PLHIV CD4 ≤ 200 cells/mm3 and asymptomatic CM infections would save U$S 11 million dollars in five years.
Finally, a review of epidemiology researches on the consequences of cryptococcal screening in asymptomatic patients highlighted a significant knowledge gap that exist worldwide, especially in Brazil. Now, the paucity of parameters is the main barrier for the development of simulation models that are more compatible with the knowledge accumulated in laboratory practice.
REFERENCES
-
1World Health Organization. Guidelines for the diagnosis, prevention and management of cryptococcal disease in HIV-infected adults, adolescents and children: supplement to the 2016 consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. Geneva: WHO; 2018. [cited 2021 Jun 22]. Available from: http://apps.who.int/iris/bitstream/handle/10665/260399/9789241550277-eng.pdf;jsessionid=3DD47D6B4F06AFA32280E2048900A316?sequence=1
» http://apps.who.int/iris/bitstream/handle/10665/260399/9789241550277-eng.pdf;jsessionid=3DD47D6B4F06AFA32280E2048900A316?sequence=1 -
2Rajasingham R, Smith RM, Park BJ, Jarvis JN, Govender NP, Chiller TM, et al. Global burden of disease of HIV-associated cryptococcal meningitis: an updated analysis. Lancet Infect Dis. 2017;17:873-81.
-
3Pappalardo MC, Melhem MS. Cryptococcosis: a review of the Brazilian experience for the disease. Rev Inst Med Trop Sao Paulo. 2003;45:299-305.
-
4Consenso em criptococose: 2008. Rev Soc Bras Med Trop. 2008;41:524-44.
-
5Vidal JE, Toniolo C, Paulino A, Colombo A, dos Anjos Martins M, da Silva Meira C, et al. Asymptomatic cryptococcal antigen prevalence detected by lateral flow assay in hospitalised HIV-infected patients in São Paulo, Brazil. Trop Med Int Heal. 2016;21:1539-44.
-
6Merry M, Boulware DR. Cryptococcal meningitis treatment strategies affected by the explosive cost of flucytosine in the United States: a cost-effectiveness analysis. Clin Infect Dis. 2016;62:1564-8.
-
7Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Vigilância, Prevenção e Controle das Infecções Sexualmente Transmissíveis, do HIV/Aids e das Hepatites Virais. Protocolo clínico e diretrizes terapêuticas para manejo da infecção pelo HIV em adultos. Brasília: Ministério da Saúde; 2018. [cited 2021 Jun 22]. Available from: http://www.aids.gov.br/pt-br/pub/2013/protocolo-clinico-e-diretrizes-terapeuticas-para-manejo-da-infeccao-pelo-hiv-em-adultos
» http://www.aids.gov.br/pt-br/pub/2013/protocolo-clinico-e-diretrizes-terapeuticas-para-manejo-da-infeccao-pelo-hiv-em-adultos -
8Vidal JE, Boulware DR. Lateral flow assay for cryptococcal antigen: an important advance to improve the continuum of HIV care and reduce cryptococcal meningitis-related mortality. Rev Inst Med Trop Sao Paulo. 2015;57 Suppl 19:38-45.
-
9Brasil. Ministério da Saúde. Indicadores e dados básicos do HIV/AIDS nos municípios brasileiros. [cited 2021 Jun 22]. Available from: http://indicadores.aids.gov.br/
» http://indicadores.aids.gov.br/ -
10Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia Diretrizes metodológicas : diretriz de avaliação econômica. 2a ed. Brasília: Ministério da Saúde; 2014. [cited 2021 Jun 22]. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/diretrizes_metodologicas_diretriz_avaliacao_economica.pdf
» https://bvsms.saude.gov.br/bvs/publicacoes/diretrizes_metodologicas_diretriz_avaliacao_economica.pdf -
11Miot J, Leong T, Takuva S, Parrish A, Dawood H. Cost-effectiveness analysis of flucytosine as induction therapy in the treatment of cryptococcal meningitis in HIV-infected adults in South Africa. BMC Health Serv Res. 2020 In Press.
-
12Brasil. Ministério da Saúde. Banco de preços em saúde. [cited 2021 Jun 22]. Available from: http://bps.saude.gov.br/login.jsf
» http://bps.saude.gov.br/login.jsf -
13Brasil. Ministério da Saúde. SIGTAP: Sistema de Gerenciamento da Tabela de Procedimentos, Medicamentos e OPM do SUS. [cited 2021 Jun 22]. Available from: http://sigtap.datasus.gov.br/tabela-unificada/app/sec/inicio.jsp
» http://sigtap.datasus.gov.br/tabela-unificada/app/sec/inicio.jsp -
14Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia, Inovação e Insumos Estratégicos em Saúde. Coordenação de Monitoramento e Avaliação de Tecnologias em Saúde. Teste diagnóstico, point of care, de Cryptococcal Antigen Lateral Flow Assay (CRAG-LFA) para detecção de infecção por Cryptococcus e diagnóstico de meningite criptocócica em pessoas vivendo com o vírus da imunodeficiência humana (PVHIV). Brasília: Ministério da Saúde; 2021. [cited 2021 Jun 22]. Available from: http://www.conitec.gov.br/images/Relatorios/2021/20210611_Relatorio_615_CragLFA_meningite-criptococica_Final.pdf
» http://www.conitec.gov.br/images/Relatorios/2021/20210611_Relatorio_615_CragLFA_meningite-criptococica_Final.pdf -
15Associação Médica Brasileira. CBHPM-2016 : Classificação Brasileira Hierarquizada de Procedimentos Médicos. Barueri: Manole; São Paulo: Associação Médica Brasileira; 2016. [cited 2021 Jun 22]. Available from: https://www.cbo.net.br/novo/publicacoes/CBHPM_2016.pdf
» https://www.cbo.net.br/novo/publicacoes/CBHPM_2016.pdf -
16Lofgren SM, Nalintya E, Meya DB, Boulware DR, Rajasingham R. A qualitative evaluation of an implementation study for cryptococcal antigen screening and treatment in Uganda. Medicine (Baltimore). 2018;97:e11722.
-
17Rajasingham R, Meya DB, Greene GS, Jordan A, Nakawuka M, Chiller TM, et al. Evaluation of a national cryptococcal antigen screening program for HIV-infected patients in Uganda: a cost-effectiveness modeling analysis. PLoS One. 2019;14:e0210105.
-
18Ramachandran A, Manabe Y, Rajasingham R, Shah M. Cost-effectiveness of CRAG-LFA screening for cryptococcal meningitis among people living with HIV in Uganda. BMC Infect Dis. 2017;17:225.
-
19Meya DB, Manabe YC, Castelnuovo B, Cook BA, Elbireer AM, Kambugu A, et al. Cost‐effectiveness of serum cryptococcal antigen screening to prevent deaths among HIV‐infected persons with a CD4 + cell count ≤100 Cells/μL who start HIV therapy in resource‐limited settings. Clin Infect Dis. 2010;51:448-55.
-
20Longley N, Jarvis JN, Meintjes G, Boulle A, Cross A, Kelly N, et al. Cryptococcal antigen screening in patients initiating ART in South Africa: a prospective cohort study. Clin Infect Dis. 2016;62:581-7.
-
21Temfack E, Rim JJ, Spijker R, Loyse A, Chiller T, Pappas PG, et al. Cryptococcal antigen in serum and cerebrospinal fluid for detecting cryptococcal meningitis in adults living with human immunodeficiency virus: systematic review and meta-analysis of diagnostic test accuracy studies. Clin Infect Dis. 2021;72:1268-78.
-
22Nalintya E, Kiggundu R, Meya D. Evolution of cryptococcal antigen testing: what is new? Curr Fungal Infect Rep. 2016;10:62-7.
-
23Boulware DR, Rolfes MA, Rajasingham R, von Hohenberg M, Qin Z, Taseera K, et al. Multisite validation of cryptococcal antigen lateral flow assay and quantification by laser thermal contrast. Emerg Infect Dis. 2014;20:45-53.
-
24Morawski BM, Boulware DR, Nalintya E, Kiragga A, Kazooza F, Rajasingham R, et al. Pre-ART cryptococcal antigen titer associated with preemptive fluconazole failure. In: Conference on Retroviruses and Opportunistic Infections; 2016 Feb 22-25; Boston, Massachusetts. [cited 2021 Jun 22]. Available from: https://www.croiconference.org/abstract/pre-art-cryptococcal-antigen-titer-associated-preemptive-fluconazole-failure/
» https://www.croiconference.org/abstract/pre-art-cryptococcal-antigen-titer-associated-preemptive-fluconazole-failure/ -
25Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia. Diretrizes metodológicas: análise de impacto orçamentário : manual para o sistema de saúde do Brasil. Brasília: Ministério da Saúde; 2012. [cited 2021 Jun 22]. Available from: https://bvsms.saude.gov.br/bvs/publicacoes/diretrizes_metodologicas_analise_impacto.pdf
» https://bvsms.saude.gov.br/bvs/publicacoes/diretrizes_metodologicas_analise_impacto.pdf -
26Instituto Brasileiro de Geografia e Estatística. Projeção da população do Brasil e das unidades da federação. [cited 2021 Jun 22]. Available from: https://www.ibge.gov.br/apps/populacao/projecao/index.html?utm_source=portal&utm_medium=popclock&utm_campaign=novo_popclock
» https://www.ibge.gov.br/apps/populacao/projecao/index.html?utm_source=portal&utm_medium=popclock&utm_campaign=novo_popclock -
27Micol R, Tajahmady A, Lortholary O, Balkan S, Quillet C, Dousset JP, et al. Cost-effectiveness of primary prophylaxis of AIDS associated cryptococcosis in Cambodia. PLoS One. 2010;5:e13856.
-
28Tenforde MW, Muthoga C, Callaghan A, Ponatshego P, Ngidi J, Mine M, et al. Cost-effectiveness of reflex laboratory-based cryptococcal antigen screening for the prevention and treatment of cryptococcal meningitis in Botswana. Wellcome Open Res. 2020;4:144.
-
29Brasil. Agência Nacional de Vigilância Sanitária. Câmara de Regulação do Mercado de Medicamentos. Preços máximos de medicamentos por princípio ativo. Brasília; 2020. [cited 2021 Jun 22]. Available from: https://www.gov.br/anvisa/pt-br/assuntos/medicamentos/cmed/precos/anos-anteriores/arquivos/4910json-file-1
» https://www.gov.br/anvisa/pt-br/assuntos/medicamentos/cmed/precos/anos-anteriores/arquivos/4910json-file-1
-
FUNDING: This project was supported by Department of Management and Incorporation of Technologies and Innovation in Health (DGITIS/SCTIE/MS) and the Pan American Health Organization Brazil – PAHO (grant Nº SCON2020-00260).
Publication Dates
-
Publication in this collection
05 July 2021 -
Date of issue
2021
History
-
Received
8 Apr 2021 -
Accepted
20 June 2021