Open-access Deglutition assessment instruments used in critical patients submitted to orotracheal extubation: a scoping review

ABSTRACT

Purpose:  to map, through a literature survey, which instruments are used to assess swallowing in patients after orotracheal extubation.

Methods:  available evidence was mapped through six electronic databases and gray literature. There were no restrictions regarding gender, ethnicity of the individuals, language of the studies, time of publication, and diagnosis.

Results:  the most mentioned protocol in the studies was the Dysphagia Risk Evaluation Protocol and the most cited objective assessment exam was the flexible endoscopic evaluation of swallowing.

Conclusion:  there is a need for a specific protocol to evaluate this profile of patients, in addition to comparative studies of subjective clinical evaluation and instrumental imaging.

Keywords: Deglutition Disorders; Intensive Care Units; Review

Introduction

Orotracheal intubation (OTI) is an invasive method commonly used in intensive care units (ICU) for respiratory assistance in critically ill hospitalized patients1. Oropharyngeal dysphagia after orotracheal extubation can occur as a result of alterations in the mechanoreceptors responsible for swallowing and lesions in the oral mucosa and in the pharyngeal, laryngeal, and tracheal regions, mainly in cases of prolonged intubation2,3.

An intubation period of more than 48 hours can be a predictor of oropharyngeal dysphagia, as approximately 14 to 56% of patients intubated for at least 48 hours present aspiration, due to swallowing disorders. In addition to pulmonary complications, altered nutrition and hydration status, due to dysphagia, can also worsen the diagnosis and intensify the risk of morbidity or mortality, besides prolonging hospitalization2,4. The Guidelines for Mechanical Ventilation propose that all patients undergoing OTI for a period of 24 hours or more undergo a speech therapy assessment aiming at the return of the oral diet and/or management of swallowing safely, acting in the prevention of aspiration pneumonia5.

The swallowing evaluation can be clinical or objective, with the use of image exams. For being fast, non-invasive, and less resource-intensive, bedside clinical assessment is the most accessible method in the daily routine of hospitals today6,7. In some cases, it is necessary to use an instrumental examination, such as evaluating swallowing, using videofluoroscopy (VFSS) or fiberoptic endoscopic evaluation of swallowing (FEES), aiming to increase diagnostic accuracy 8-10. No mapping of instruments used to assess post-extubation dysphagia in critically ill patients was found in the literature.

Thus, this scoping review aimed at mapping and synthesizing the scientific evidence available on the instruments for assessing swallowing, after orotracheal extubation, in critically ill patients.

Methods

Protocol and Registration

This scoping review was developed according to the Preferred Reporting Items for Systematic reviews and Meta- Analyses Extension for Scoping Reviews (PRISMA-ScR)11 and was registered on the Open Science Framework platform (doi: 10.17605/OSF.IO/Q8KZB).

Eligibility Criteria

The acronym ‘PCC’ was used to consider the eligibility of studies for this review, standing for:

P = Population (≥18 years old).

C = Concept (screening instruments, clinical evaluation protocol and imaging instruments).

C = Context (critically ill patients admitted to the intensive care unit).

Inclusion Criteria

To map studies with a higher level of evidence, only primary and analytical studies were included, such as clinical trials, cohort, case-control, cross- sectional, prospective or retrospective studies which assessed instrumental and/or clinical swallowing in critically ill adult or elderly patients undergoing orotracheal extubation. There were no restrictions regarding gender, ethnicity of the individuals, language of the studies, time of publication, and diagnosis.

Exclusion Criteria

The following exclusion criteria were applied: a) Reviews, case reports, personal opinions, letters, posters, and conference abstracts; b) studies with children’s population; c) studies with tracheostomized patients after orotracheal extubation; d) under 18 years old; e) animal studies; f) studies that did not assess the outcome of interest or tthat presented incomplete data.

Information Sources and Search

Word combinations were adapted for each of the six electronic databases selected as the sources for the search, namely: EMBASE, Latin American and Caribbean Health Sciences Literature (LILACS), Livivo, PubMed/Medline, Scopus, and Web of Science. In addition, grey literature was also used as information source through AshaWire, Google Scholar, Open Grey, and ProQuest (see, Appendix 1 Appendix 1: Database search strategy Lilacs («Intubação intratraqueal» OR «intubações intratraqueais» OR «intubação endotraqueal» OR «intubações endotraqueais» OR «Extubação das vias aéreas» OR «extubações das vias aéreas» OR «Extubação traqueal» OR «Extubações traqueais» OR «Extubação intratraqueal» OR «Extubações intratraqueais» OR «Extubação endotraqueal» OR «Extubações endotraqueais» OR «Intubación intratraqueal» OR «intubaciones intratraqueales» OR «intubación endotraqueal» OR «intubaciones endotraqueales» OR «Extubación de la vía aérea» OR «extubaciones de vías aéreas» OR «Extubación traqueal» OR «extubaciones traqueales» OR «Extubación intratraqueal» OR «extubaciones intratraqueales» OR «Extubación intratraqueal» OR «Extubación endotraqueal» OR «extubaciones endotraqueales» OR «intratracheal intubation» OR «intratracheal intubations» OR «endotracheal intubation» OR «endotracheal intubations» OR «Airway Extubation» OR «airway extubations» OR «Tracheal Extubation» OR «Tracheal Extubations» OR «Intratracheal Extubation» OR «Intratracheal Extubations» OR «Endotracheal Extubation» OR «Endotracheal Extubations») AND («Trastornos de la deglución» OR «Trastorno de la deglución» OR «transtorno de deglución» OR «transtornos de deglución» OR «Disfagia» OR «Disfagia orofaríngea» OR «Disfagia esofágica» OR «Transtornos da Deglutição» OR «Transtorno da Deglutição» OR «Disfagia» OR «Disfagia orofaríngea» OR «Disfagia esofágica» OR «Deglutition Disorders» OR «Deglutition Disorder» OR «Swallowing Disorders» OR «Swallowing Disorder» OR «Dysphagia» OR «Oropharyngeal Dysphagia» OR «Esophageal Dysphagia») AND («Patología del habla y Lenguaje» OR «Patología del lenguaje» OR «Patología del habla» OR «Rehabilitación de trastornos del habla y lenguaje» OR «Rehabilitación de trastorno del habla y lenguaje» OR «Rehabilitación del trastorno del lenguaje y del habla» OR «Rehabilitación del trastornos del habla y lenguaje» OR «fonoaudiologia» OR «patologia da fala e linguagem» OR «Patologia da Linguagem» OR «Patologia da Fala» OR «Reabilitação de Distúrbios da Fala e da Linguagem» OR «Reabilitação de Distúrbios de Linguagem e Fala» OR «Reabilitação de Distúrbios de Fala e Linguagem» OR «Speech-Language Pathology» OR «Speech Language Pathology» OR «Language Pathology» OR «Speech Pathology» OR «Rehabilitation of Speech and Language Disorders» OR «Language and Speech Disorder Rehabilitation» OR «Speech and Language Disorder Rehabilitation») PubMed 1. ((“intubation, intracheal”[MeSH Terms]) OR (“intubation, intracheal”[All Fields]) OR (“intratracheal intubation”[All Fields]) OR (“intratracheal intubations”[All Fields]) OR (“endotracheal intubation”[All Fields]) OR (“endotracheal intubations”[All Fields]) OR (“Airway Extubation”[MeSH Terms]) OR (“Airway Extubation”[All Fields]) OR (“airway extubations”[All Fields]) OR (“Tracheal Extubation”[All Fields]) OR (“Tracheal Extubations”[All Fields]) OR (“Intratracheal Extubation”[All Fields]) OR (“Intratracheal Extubations”[All Fields]) OR (“Endotracheal Extubation”[All Fields]) OR (“Endotracheal Extubations”[All Fields])) 2. ((“Deglutition Disorders”[MeSH Terms]) OR (“Deglutition Disorders”[All Fields]) OR (“Deglutition Disorder”[All Fields]) OR (“Swallowing Disorders”[All Fields]) OR (“Swallowing Disorder”[All Fields]) OR (“Dysphagia”[All Fields]) OR (“Oropharyngeal Dysphagia”[All Fields]) OR (“Esophageal Dysphagia”[All Fields])) 3. ((“Speech-Language Pathology”[MeSH Terms]) OR (“Speech-Language Pathology”[All Fields]) OR (“Speech Language Pathology”[All Fields]) OR (“Language Pathology”[All Fields]) OR (“Speech Pathology”[All Fields]) OR (“Rehabilitation of Speech and Language Disorders”[MeSH Terms]) OR (“Rehabilitation of Speech and Language Disorders”[All Fields]) OR (“Language and Speech Disorder Rehabilitation”[All Fields]) OR (“Speech and Language Disorder Rehabilitation”[All Fields])) 4. #1 AND #2 AND #3 SCOPUS (“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) AND (“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) AND (“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”) Web of Science 1. TS=(“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) 2. TS=(“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) 3. TS=(“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”) 4. #1 AND #2 AND #3 Embase (‘intratracheal intubation’/exp OR ‘intratracheal intubation’ OR ‘intratracheal intubations’ OR ‘endotracheal intubation’/exp OR ‘endotracheal intubation’ OR ‘endotracheal intubations’ OR ‘airway extubation’/exp OR ‘airway extubation’ OR ‘airway extubations’ OR ‘tracheal extubation’/exp OR ‘tracheal extubation’ OR ‘tracheal extubations’ OR ‘intratracheal extubation’ OR ‘intratracheal extubations’ OR ‘endotracheal extubation’/exp OR ‘endotracheal extubation’ OR ‘endotracheal extubations’) AND (‘deglutition disorders’/exp OR ‘deglutition disorders’ OR ‘deglutition disorder’/exp OR ‘deglutition disorder’ OR ‘swallowing disorders’ OR ‘swallowing disorder’/exp OR ‘swallowing disorder’ OR ‘dysphagia’/exp OR ‘dysphagia’ OR ‘oropharyngeal dysphagia’/exp OR ‘oropharyngeal dysphagia’ OR ‘esophageal dysphagia’/exp OR ‘esophageal dysphagia’) AND (‘speech-language pathology’/exp OR ‘speech-language pathology’ OR ‘speech language pathology’/exp OR ‘speech language pathology’ OR ‘language pathology’ OR ‘speech pathology’ OR ‘rehabilitation of speech and language disorders’/exp OR ‘rehabilitation of speech and language disorders’ OR ‘language and speech disorder rehabilitation’ OR ‘speech and language disorder rehabilitation’) Livivo TI=(“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) AND TI=(“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) AND TI=(“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”) AshaWire (“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) AND (“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) AND (“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”) Google Scholar “Endotracheal Extubations” AND “deglutition disorders” Open Grey “Endotracheal Extubations” ProQuest NOFT(“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) AND NOFT(“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) AND NOFT(“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”) ).

Searches of electronic databases and gray literature were performed on June 20, 2020 and updated on May 26, 2021. All references were managed and all duplicate studies were removed using an appropriate software (EndNote® X7 Thomson Reuters, Philadelphia, PA).

References were manually searched in all included studies and in the most current guidelines in the literature that have addressed instruments used to assess swallowing in extubated patients.

Selection of Sources of Evidence

The selection of articles was carried out in two phases. In the first phase, two reviewers (R.D.S and R.S.S) independently reviewed the titles and abstracts of all references. All articles that did not meet the previously established criteria were excluded at this stage. In the second phase, the same reviewers read the full text of the articles selected in the first phase, also independently. When there was no consensus even after discussion, a third reviewer (K.V.M.T) was involved for the final decision.

To facilitate reading independently, the Rayyan website (http://rayyan.qcri.org) was used. Besides the two reviewers who performed the assessments blindly, a third member of the team (K.V.M.T) acted as moderator.

Data Charting Process and Data Items

The data collected consisted of study characteristics (author, year of publication, country, and study design), population characteristics (gender, age, and pathology), assessment instruments, and outcome.

If the required data were not complete, efforts were made to contact the authors to obtain any unpublished data. The authors could be contacted by email for three consecutive weeks in search of more information.

All information related to the instruments for evaluating swallowing in patients after orotracheal extubation was extracted and mapped. As this is a descriptive review, any measure of effect was considered and used in the qualitative synthesis.

Reporting Bias

To reduce the likelihood of reporting bias, a broad search strategy was carried out through five electronic databases, including a non-English-language database (LILACS). In addition, a search was also carried out in the grey literature to verify the existence of studies that met the eligibility criteria, but which had not been published.

Results

Study Selection

The flow of studies through the scoping review process is shown in Figure 1. A total of 471 articles were retrieved from the five electronic databases. After removing duplicate articles, 430 references were maintained. Subsequently, after applying the eligibility criteria, 413 studies were excluded, resulting in 17 articles. A search was carried out in the grey literature and in the reference list of articles, thus, totaling 23 studies for complete reading. After the complete reading (second phase), 11 articles were excluded (see Appendix 2 Appendix 2: Reason for excluded studies Author, Year Reason for Exclusion Barquist E, Brown M, Cohn S, Lundyv D, Jackowski J. 2001¹ 3 Bordon A, Bokhari R, Sperry J, Testa IVD, Feinstein A, Ghaemmaghami V. 2011² 1 Cheung W, Clayton N, Li F, Tan J, Milliss D, Thanakrishnan G, Maitz P. 2013³ 1 Daly E, Miles A, Scott S, Gillham M. 2016⁴ 2 Johnson KL, Speirs L, Mitchell A, Przybyl H, Anderson D, Manos B., et al. 20185 1 Macht M, King, CJ, Wimbish T, Clark BJ, Benson AB, Burnham EL., et al. 20136 2 Macht M, Wimbish T, Clark BJ, Benson AB, Burnham EL, Williams A., et al. 20127 1 Malandraki GA., Markaki V, Georgopoulos VC, Psychogios L, Nanas S. 20168 1 Partik B, Pokieser P, Schima W, Schober E, Stadler A, Eisenhuber E., et al. 20009 2 Regan J, Walshe M, Lavan S, Horan E, Gillivan-Murphy P, Healy A., et al. 202110 2 Skoretz SA, Yau TM., Ivanov J, Granton JT, Martino R. 201411 1 1. Study that did not specify the swallowing assessment instrument; 2. Study that included patients with tracheostomy; 3. Study that included patients under 18 years of age. References 1. Barquist E, Brown M, Cohn S, Lundyv D, Jackowski J. Postextubation fiberoptic endoscopic evaluation of swallowing after prolonged endotracheal intubation: a randomized, prospective trial. Crit Care Med. 2001;29(9):1710-3. 2. Bordon A, Bokhari R, Sperry J, Testa IVD, Feinstein A, Ghaemmaghami V. Swallowing dysfunction after prolonged intubation: analysis of risk factors in trauma patients. Am J Surg. 2011;202(6):679-83. 3. Cheung W, Clayton N, Li F, Tan J, Milliss D, Thanakrishnan G et al. The effect of endotracheal tube size on voice and swallowing function in patients with thermal burn injury: an evaluation using the Australian Therapy Outcome Measures (AusTOMS). Int J Speech Lang Pathol. 2013;15(2):216-20. 4. Daly E, Miles A, Scott S, Gillham M. Finding the red flags: swallowing difficulties after cardiac surgery in patients with prolonged intubation. J Crit Care. 2016;31(1):119-24. 5. Johnson KL, Speirs L, Mitchell A, Przybyl H, Anderson D, Manos B et al. Validation of a postextubation dysphagia screening tool for patients after prolonged endotracheal intubation. J Crit Care. 2018;27(2):89-96. 6. Macht M, King CJ, Wimbish T, Clark BJ, Benson AB, Burnham EL et al. Postextubation dysphagia is associated with longer hospitalization in survivors of critical illness with neurologic impairment. J Crit Care. 2013;17(3):1-9. 7. Macht M, Wimbish T, Clark BJ, Benson AB, Burnham EL, Williams A et al. Diagnosis and treatment of post-extubation dysphagia: results from a national survey. J Crit Care. 2012;27(6):578-86. 8. Malandraki GA, Markaki V, Georgopoulos VC, Psychogios L, Nanas S. Postextubation dysphagia in critical patients: a first report from the largest step-down intensive care unit in Greece. Int J Speech Lang Pathol. 2016;25(2):150-6. 9. Partik B, Pokieser P, Schima W, Schober E, Stadler A, Eisenhuber E et al. Videofluoroscopy of swallowing in symptomatic patients who have undergone long-term intubation. AJR. 2000;174(5):1409-12. 10. Regan J, Walshe M, Lavan S, Horan E, Gillivan-Murphy P, Healy A et al. Postextubation dysphagia and dysphonia amongst adults with COVID-19 in the Republic of Ireland: a prospective multi-site observational cohort study. Clin Otolaryngol. 2021;46(6):1290-9. 11. Skoretz SA, Yau TM, Ivanov J, Granton JT, Martino R. Dysphagia and associated risk factors following extubation in cardiovascular surgical patients. Dysphagia. 2014;29(6):647-54. ). In the search update, two articles were added, resulting in a total of 14 studies included for qualitative synthesis and mapping of results.

Figure 1:
Flowchart of literature search and selection criteria

Study Characteristics

The studies included were published from 2003 12 to 202013), (14 and conducted in Brazil1), (15-21, in the United States of America (USA)12-14), (22), (23, and in Canada24. The sample size of the studies ranged from 324 to 21313), (14 participants aged between 18 and 90 years15, with a higher prevalence of males.

All studies used a clinical assessment method based on protocols and/or objective imaging instruments. The outcome assessed in all studies was the applicability of different protocols and imaging exams related to the purpose of the study, but with great variability and without standardization. Two studies performed both forms of assessment16), (23. The most used methods to assess swallowing after orotracheal extubation were the Dysphagia Risk Evaluation Protocol (PARD)25 and the FEES (Figure 2). As for the design of the studies, cross-sectional observational, cohort studies and two non-randomized clinical trials were found. For all the included studies, descriptive characteristics are shown in Table 1.

Figure 2:
Radial bar chart for the frequency of the tool used to assess swallowing in patients after orotracheal extubation, and characteristics of the tools used

Table 1:
Summary of descriptive characteristics and outcomes of interest of the included studies (n=14)

Results of Individual Sources of Evidence

PARD was the most used protocol in the studies found16-21. It was considered efficient in the identification of clinical signs suggestive of bronchoaspiration, due to its wide range of evaluated aspects, making it possible to achieve the objective of the studies. Some authors emphasize the importance and refer to the absence of an objective imaging exam to compare the results as a limitation of their studies17), (22), (21.

The applicability of PARD was also considered effective in conjunction with other instruments, such as the Preliminary Assessment Protocol (PAP), which allows the assessor to carry out a more complete assessment with offers in different consistencies and the Protocol for the Introduction and Transition of Oral Feeding (PITA, acronym in Portuguese), which establishes the description of levels of oral diet and fluid consistency9. Developed as a screening tool for identifying eating and swallowing disorders in patients with stroke, for patients with a neurological profile affected by cerebrovascular accident (CVA), the Mann Assessment of Swallowing Ability (MASA) was used for re-search in patients with varied diagnoses, and considered effective by the author for the purposes of the study15. The execution of the bedside evaluation, in conjunction with the surface electromyography image exam, associated with the use of PARD, was effective in relation to the evaluation and therapeutics16.

The Yale Swallow screening protocol26 for aspiration risk was associated with the use of FEES only in patients who failed to have the protocol applied to them23. As a reference, in a study with patients diagnosed with Acute Respiratory Distress Syndrome (ARDS), the VFSS was used and considered effective for the defined objective, thus contributing with a more objective analysis of swallowing, and expanding the knowledge of the physiological aspects of swallowing in patients with ARDS18.

In another study15, the methodological goal of joining the objective imaging tests VFSS and FEES was not achieved, as patients refused to undergo a FEES, as it is considered an invasive method, giving preference to the performance of VFSS. In a controlled study, there was a suggestion to use the FEES, aiming at creating a standard protocol for patients undergoing orotracheal intubation. In addition, it was observed that FEES did not prevent aspiration pneumonia. Therefore, a randomized study was suggested to verify the effectiveness of the test as a method of preventing aspiration in these patients1. The use of FEES is indicated to be performed before the bedside clinical functional assessment, due to its effectiveness in identifying silent aspiration and the possibility of visualizing abnormalities caused by OTI12), (13.

Discussion

This scoping review mapped the scientific evidence available on the instruments for assessing swallowing in patients after orotracheal extubation. There is a prevalence of approximately 44 to 87% of swallowing dysfunction in patients that underwent post-tracheal extubation. Such alteration is called dysphagia and is characterized by any alteration of neuronal or structural aspect that alters the correct swallowing process, reducing patient safety4), (27.

The speech-language assessment of swallowing after orotracheal extubation is based on the execution of protocols and/or objective imaging exams. PARD, the most cited protocol among the studies, was created based on the theoretical basis of the clinical aspects most observed in the literature. It is aimed objectively to the functional assessment with the supply of water and pasty foods, being used for various diagnoses25.

In addition, it covers a range of aspects necessary to interpret and identify dysphagia and helps to identify the clinical signs of bronchoaspiration and the approach to be taken based on the results obtained. Its applicability was also considered effective in conjunction with other protocols, such as the PITA20. Each protocol exposes functions for different moments of the speech-language assessment, needing attention to the subjectivity of the interpretation of the combination of results and the objective that the professional seeks at the moment. Knowing that a good methodological basis can positively influence the reduction of the incidence of aspiration pneumonia in hospitalized patients27, some authors confirm in their studies that the aspects evaluated in PARD are associated with what the literature confirms to be predictors of dysphagia in patients with a long period of orotracheal intubation 22.

Thus, it is possible to observe that such literary and practical relevance influenced the use of this protocol in most studies, fulfilling the main objective of the authors. The MASA protocol, aimed at neurological patients after stroke, assesses similar aspects of other protocols, but also considers some functions related to the cranial nerves. With a sensitivity index of approximately 93%, it was also referred as an assessment tool in post-OTI patients, including varied and some non-neurological diagnoses15), (28.

It is believed that its use in association with imaging exams would facilitate the objectivity of the evaluative outcome, providing a better approach in some cases. Only two studies16), (23 managed to combine the two forms. One of them is the association of FEES with the Yale Swallow Protocol, validated with the use of VFSS as a standard reference, with approximately 100% sensitivity16).

Other authors also commented on the benefit of combining objective and subjective evaluations20-22, with the use of the FEES1), (4), (12-14), (23 and VFSS18), (24 which were the most cited imaging exams as a reference for the speech therapy assessment of swallowing.

The FEES assessment has as a facilitating agent, which is its performance at the bedside, without needing movement from the patient. However, it was said that there is no need to perform it in all extubated patients, but in groups that fail in some aspect during the clinical assessment at the bedside15. In addition, it was said that its use, despite being of important relevance in aiding speech therapy assessments, does not confirm the absence of risk of aspiration pneumonia29.

The VFSS stands out for being a non-invasive test with better reception by the patients, besides expanding the understanding of the swallowing process. However, its applicability in studies is reduced due to the high investment cost for the examination, the radiological exposure of the patient and professionals, and the impossibility of its performance at the bedside18), (24.

Thus, the speech therapist who works in a hospital environment, with the task of preventing and reducing complications caused by dysphagia30, benefits from well- structured protocols and objective exams, which guide a quality and evidence-based speech therapy conduct31.

As a research limitation, studies with a diversity of underlying diseases and several forms of assessment, with different protocols often not specified in the methodology, making it difficult to identify and standardize the evaluation, can be cited.

Conclusion

No standard protocol for the assessment of extubated patients was found in the literature. In the mapping performed in this research, the most used protocol for evaluation was PARD and the imaging exam was FEES.

REFERENCES

  • 1 Kunigk MRG, Chehter E. Disfagia orofaríngea em pacientes submetidos à entubação orotraqueal. Rev Soc Bras Fonoaudiol. 2007;12(4):287-91.
  • 2 Brodsky MB, González-Fernández M, Mendez-Tellez PA, Shanholtz C, Palmer JB, Needham DM. Factors associated with swallowing assessment after oral endotracheal intubation and mechanical ventilation for acute lung injury. Ann Am Thorac Soc. 2014;11(10):1545-52.
  • 3 Goldsmith T. Evaluation and treatment of swallowing disorders following endotracheal intubation and tracheostomy. Int Anesthesiol Clin. 2000;38(3):219-42.
  • 4 Skoretz SA, Flowers HL, Martino R. The incidence of dysphagia following endotracheal intubation: a systematic review. Chest. 2010;137(3):665-73.
  • 5 Barbas CSV, Ísola AM, Farias AMDC, Cavalcanti AB, Gama AMC, Duarte ACM et al. Recomendações brasileiras de ventilação mecânica 2013. Parte I. Rev Bras Ter Intensiva. 2014;26(2):89-121.
  • 6 Mann G. Effectiveness and efficiency: state of the art in dysphagia rehabilitation after stroke. Aust Comm Quarterly. 1996;4:25-7.
  • 7 Carnaby-Mann G, Lenius K. The bedside examination in dysphagia. Phys Med Rehabil Clin. 2008;19(4):747-68.
  • 8 Mathers-Schmidt BA, Kurlinski M. Dysphagia evaluation practices: inconsistencies in clinical assessment and instrumental examination decision-making. Dysphagia. 2003;18(2):114-25.
  • 9 Padovani ARP. Oral Feeding Transition Protocol (OFTP) [Dissertation]. Sao Paulo (SP): Faculty of Medicine, University of São Paulo; 2010.
  • 10 de Lima Amaral IJ, de Castro Guimarães V, Diniz DS, Carneiro MAD. Clinical swallowing evaluation in multiple sclerosis at a reference center in the center-west of Brazil. Braz J Dev. 2020;6(7):51166-78.
  • 11 Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467-73.
  • 12 El Solh A, Okada M, Bhat A, Pietrantoni C. Swallowing disorders post orotracheal intubation in the elderly. Intensive Care Med. 2003;29(9):1451-5.
  • 13 Langmore SE, Krisciunas GP, Warner H, White D, Dvorkin D, Fink D et al. Abnormalities of aspiration and swallowing function in survivors of acute respiratory failure. Dysphagia. 2021;36(5):831-41.
  • 14 Moss M, White SD, Warner H, Dvorkin D, Fink D, Gomez-Taborda S et al. Development of an accurate bedside swallowing evaluation decision tree algorithm for detecting aspiration in acute respiratory failure survivors. Chest. 2020;158(5):1923-33.
  • 15 Oliveira ACMD, Friche AADL, Salomão MS, Bougo GC, Vicente LCC. Predictive factors for oropharyngeal dysphagia after prolonged orotracheal intubation. Braz J Otorhinolaryngol. 2018;84(6):722-8.
  • 16 El Gharib AZG, Berretin-Felix G, Rossoni DF, Yamada SS. Effectiveness of therapy on post- extubation dysphagia: clinical and electromyographic findings. Clin Med Insights Ear Nose Throat. 2019;12:1179550619873364
  • 17 Moraes D, Sassi F, Mangilli L, Zilberstein B, de Andrade CRF. Clinical prognostic indicators of dysphagia following prolonged orotracheal intubation in ICU patients. J Crit care. 2013;17(5):1-10.
  • 18 Medeiros GCD, Sassi FC, Zambom LS, de Andrade CRF. Correlation between the severity of critically ill patients and clinical predictors of bronchial aspiration. J Bras Pneumol. 2016;42(2):114-20.
  • 19 Medeiros GC, Sassi FC, Mangilli LD, Zilberstein B, Andrade CR. Clinical dysphagia risk predictors after prolonged orotracheal intubation. Clinics. 2014;69(1):8-14.
  • 20 Padovani AR, Moraes DP, Sassi FC, de Andrade CRF. Clinical swallowing assessment in intensive care unit. Rev Soc Bras Fonoaudiol. 2013;35(1):1-7.
  • 21 Ferrucci JL, Sassi FC, Medeiros GCD, de Andrade CRF. Comparison between the functional aspects of swallowing and clinical markers in ICU patients with Traumatic Brain Injury (TBI). CoDAS. 2019;31(2):e20170278.
  • 22 Brodsky MB, De I, Chilukuri K, Huang M, Palmer BJ, Needham DM. Coordination of pharyngeal and laryngeal swallowing events during single liquid swallows after oral endotracheal intubation for patients with acute respiratory distress syndrome. Dysphagia. 2018;33(6):768-77.
  • 23 Leder SB, Warner HL, Suiter DM, Young NO, Bhattacharya B, Siner JM et al. Evaluation of swallow function post-extubation: is it necessary to wait 24 hours? Ann Otol Rhinol. 2019;128(7):619-24.
  • 24 Skoretz SA, Yau TM, Granton JT, Martino R. The feasibility of assessing swallowing physiology following prolonged intubation after cardiovascular surgery. Pilot Feasibility Stud. 2017;3(1):1-10.
  • 25 Padovani AR, Moraes DP, Mangili LD, de Andrade CRF. Protocolo fonoaudiológico de avaliação do risco para disfagia (PARD). Rev Soc Bras Fonoaudiol. 2007;12(3):199-205.
  • 26 Suiter DM, Sloggy J, Leder SB. Validation of the Yale Swallow Protocol: a prospective double-blinded videofluoroscopy study. Dysphagia. 2014;29(2):199-203.
  • 27 Leder SB, Cohn SM, Moller BA. Fiberoptic endoscopic documentation of the high incidence of aspiration following extubation in critically ill trauma patients. Dysphagia. 1998;13(4):208-12.
  • 28 Mann G. MASA: The Mann Assessment of Swallowing Ability. 2002; Vol. 1. Cengage learning.
  • 29 Barquist E, Brown M, Cohn S, Lundy D, Jackowski J. Postextubation fiberoptic endoscopic evaluation of swallowing after prolonged endotracheal intubation: a randomized, prospective trial. Crit Care Med. 2001;29(9):1710-3.
  • 30 Lin LC, Wu SC, Chen HS, Wang TG, Chen MY. Prevalence of impaired swallowing in institutionalized older people in Taiwan. J Am Geriatr Soc. 2002;50(6):1118-23.
  • 31 Apel K, Self T. Evidence-based practice: the marriage of research and clinical services. The ASHA Leader. 2003;8(16):6-7.

Appendix 1: Database search strategy

Lilacs («Intubação intratraqueal» OR «intubações intratraqueais» OR «intubação endotraqueal» OR «intubações endotraqueais» OR «Extubação das vias aéreas» OR «extubações das vias aéreas» OR «Extubação traqueal» OR «Extubações traqueais» OR «Extubação intratraqueal» OR «Extubações intratraqueais» OR «Extubação endotraqueal» OR «Extubações endotraqueais» OR «Intubación intratraqueal» OR «intubaciones intratraqueales» OR «intubación endotraqueal» OR «intubaciones endotraqueales» OR «Extubación de la vía aérea» OR «extubaciones de vías aéreas» OR «Extubación traqueal» OR «extubaciones traqueales» OR «Extubación intratraqueal» OR «extubaciones intratraqueales» OR «Extubación intratraqueal» OR «Extubación endotraqueal» OR «extubaciones endotraqueales» OR «intratracheal intubation» OR «intratracheal intubations» OR «endotracheal intubation» OR «endotracheal intubations» OR «Airway Extubation» OR «airway extubations» OR «Tracheal Extubation» OR «Tracheal Extubations» OR «Intratracheal Extubation» OR «Intratracheal Extubations» OR «Endotracheal Extubation» OR «Endotracheal Extubations») AND («Trastornos de la deglución» OR «Trastorno de la deglución» OR «transtorno de deglución» OR «transtornos de deglución» OR «Disfagia» OR «Disfagia orofaríngea» OR «Disfagia esofágica» OR «Transtornos da Deglutição» OR «Transtorno da Deglutição» OR «Disfagia» OR «Disfagia orofaríngea» OR «Disfagia esofágica» OR «Deglutition Disorders» OR «Deglutition Disorder» OR «Swallowing Disorders» OR «Swallowing Disorder» OR «Dysphagia» OR «Oropharyngeal Dysphagia» OR «Esophageal Dysphagia») AND («Patología del habla y Lenguaje» OR «Patología del lenguaje» OR «Patología del habla» OR «Rehabilitación de trastornos del habla y lenguaje» OR «Rehabilitación de trastorno del habla y lenguaje» OR «Rehabilitación del trastorno del lenguaje y del habla» OR «Rehabilitación del trastornos del habla y lenguaje» OR «fonoaudiologia» OR «patologia da fala e linguagem» OR «Patologia da Linguagem» OR «Patologia da Fala» OR «Reabilitação de Distúrbios da Fala e da Linguagem» OR «Reabilitação de Distúrbios de Linguagem e Fala» OR «Reabilitação de Distúrbios de Fala e Linguagem» OR «Speech-Language Pathology» OR «Speech Language Pathology» OR «Language Pathology» OR «Speech Pathology» OR «Rehabilitation of Speech and Language Disorders» OR «Language and Speech Disorder Rehabilitation» OR «Speech and Language Disorder Rehabilitation») PubMed 1. ((“intubation, intracheal”[MeSH Terms]) OR (“intubation, intracheal”[All Fields]) OR (“intratracheal intubation”[All Fields]) OR (“intratracheal intubations”[All Fields]) OR (“endotracheal intubation”[All Fields]) OR (“endotracheal intubations”[All Fields]) OR (“Airway Extubation”[MeSH Terms]) OR (“Airway Extubation”[All Fields]) OR (“airway extubations”[All Fields]) OR (“Tracheal Extubation”[All Fields]) OR (“Tracheal Extubations”[All Fields]) OR (“Intratracheal Extubation”[All Fields]) OR (“Intratracheal Extubations”[All Fields]) OR (“Endotracheal Extubation”[All Fields]) OR (“Endotracheal Extubations”[All Fields])) 2. ((“Deglutition Disorders”[MeSH Terms]) OR (“Deglutition Disorders”[All Fields]) OR (“Deglutition Disorder”[All Fields]) OR (“Swallowing Disorders”[All Fields]) OR (“Swallowing Disorder”[All Fields]) OR (“Dysphagia”[All Fields]) OR (“Oropharyngeal Dysphagia”[All Fields]) OR (“Esophageal Dysphagia”[All Fields])) 3. ((“Speech-Language Pathology”[MeSH Terms]) OR (“Speech-Language Pathology”[All Fields]) OR (“Speech Language Pathology”[All Fields]) OR (“Language Pathology”[All Fields]) OR (“Speech Pathology”[All Fields]) OR (“Rehabilitation of Speech and Language Disorders”[MeSH Terms]) OR (“Rehabilitation of Speech and Language Disorders”[All Fields]) OR (“Language and Speech Disorder Rehabilitation”[All Fields]) OR (“Speech and Language Disorder Rehabilitation”[All Fields])) 4. #1 AND #2 AND #3 SCOPUS (“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) AND (“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) AND (“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”) Web of Science 1. TS=(“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) 2. TS=(“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) 3. TS=(“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”) 4. #1 AND #2 AND #3 Embase (‘intratracheal intubation’/exp OR ‘intratracheal intubation’ OR ‘intratracheal intubations’ OR ‘endotracheal intubation’/exp OR ‘endotracheal intubation’ OR ‘endotracheal intubations’ OR ‘airway extubation’/exp OR ‘airway extubation’ OR ‘airway extubations’ OR ‘tracheal extubation’/exp OR ‘tracheal extubation’ OR ‘tracheal extubations’ OR ‘intratracheal extubation’ OR ‘intratracheal extubations’ OR ‘endotracheal extubation’/exp OR ‘endotracheal extubation’ OR ‘endotracheal extubations’) AND (‘deglutition disorders’/exp OR ‘deglutition disorders’ OR ‘deglutition disorder’/exp OR ‘deglutition disorder’ OR ‘swallowing disorders’ OR ‘swallowing disorder’/exp OR ‘swallowing disorder’ OR ‘dysphagia’/exp OR ‘dysphagia’ OR ‘oropharyngeal dysphagia’/exp OR ‘oropharyngeal dysphagia’ OR ‘esophageal dysphagia’/exp OR ‘esophageal dysphagia’) AND (‘speech-language pathology’/exp OR ‘speech-language pathology’ OR ‘speech language pathology’/exp OR ‘speech language pathology’ OR ‘language pathology’ OR ‘speech pathology’ OR ‘rehabilitation of speech and language disorders’/exp OR ‘rehabilitation of speech and language disorders’ OR ‘language and speech disorder rehabilitation’ OR ‘speech and language disorder rehabilitation’) Livivo TI=(“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) AND TI=(“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) AND TI=(“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”) AshaWire (“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) AND (“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) AND (“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”) Google Scholar “Endotracheal Extubations” AND “deglutition disorders” Open Grey “Endotracheal Extubations” ProQuest NOFT(“intratracheal intubation” OR “intratracheal intubations” OR “endotracheal intubation” OR “endotracheal intubations” OR “Airway Extubation” OR “airway extubations” OR “Tracheal Extubation” OR “Tracheal Extubations” OR “Intratracheal Extubation” OR “Intratracheal Extubations” OR “Endotracheal Extubation” OR “Endotracheal Extubations”) AND NOFT(“Deglutition Disorders” OR “Deglutition Disorder” OR “Swallowing Disorders” OR “Swallowing Disorder” OR “Dysphagia” OR “Oropharyngeal Dysphagia” OR “Esophageal Dysphagia”) AND NOFT(“Speech-Language Pathology” OR “Speech Language Pathology” OR “Language Pathology” OR “Speech Pathology” OR “Rehabilitation of Speech and Language Disorders” OR “Language and Speech Disorder Rehabilitation” OR “Speech and Language Disorder Rehabilitation”)

Appendix 2: Reason for excluded studies

Author, Year Reason for Exclusion Barquist E, Brown M, Cohn S, Lundyv D, Jackowski J. 2001¹ 3 Bordon A, Bokhari R, Sperry J, Testa IVD, Feinstein A, Ghaemmaghami V. 2011² 1 Cheung W, Clayton N, Li F, Tan J, Milliss D, Thanakrishnan G, Maitz P. 2013³ 1 Daly E, Miles A, Scott S, Gillham M. 2016⁴ 2 Johnson KL, Speirs L, Mitchell A, Przybyl H, Anderson D, Manos B., et al. 20185 1 Macht M, King, CJ, Wimbish T, Clark BJ, Benson AB, Burnham EL., et al. 20136 2 Macht M, Wimbish T, Clark BJ, Benson AB, Burnham EL, Williams A., et al. 20127 1 Malandraki GA., Markaki V, Georgopoulos VC, Psychogios L, Nanas S. 20168 1 Partik B, Pokieser P, Schima W, Schober E, Stadler A, Eisenhuber E., et al. 20009 2 Regan J, Walshe M, Lavan S, Horan E, Gillivan-Murphy P, Healy A., et al. 202110 2 Skoretz SA, Yau TM., Ivanov J, Granton JT, Martino R. 201411 1 1. Study that did not specify the swallowing assessment instrument; 2. Study that included patients with tracheostomy; 3. Study that included patients under 18 years of age. References 1. Barquist E, Brown M, Cohn S, Lundyv D, Jackowski J. Postextubation fiberoptic endoscopic evaluation of swallowing after prolonged endotracheal intubation: a randomized, prospective trial. Crit Care Med. 2001;29(9):1710-3. 2. Bordon A, Bokhari R, Sperry J, Testa IVD, Feinstein A, Ghaemmaghami V. Swallowing dysfunction after prolonged intubation: analysis of risk factors in trauma patients. Am J Surg. 2011;202(6):679-83. 3. Cheung W, Clayton N, Li F, Tan J, Milliss D, Thanakrishnan G et al. The effect of endotracheal tube size on voice and swallowing function in patients with thermal burn injury: an evaluation using the Australian Therapy Outcome Measures (AusTOMS). Int J Speech Lang Pathol. 2013;15(2):216-20. 4. Daly E, Miles A, Scott S, Gillham M. Finding the red flags: swallowing difficulties after cardiac surgery in patients with prolonged intubation. J Crit Care. 2016;31(1):119-24. 5. Johnson KL, Speirs L, Mitchell A, Przybyl H, Anderson D, Manos B et al. Validation of a postextubation dysphagia screening tool for patients after prolonged endotracheal intubation. J Crit Care. 2018;27(2):89-96. 6. Macht M, King CJ, Wimbish T, Clark BJ, Benson AB, Burnham EL et al. Postextubation dysphagia is associated with longer hospitalization in survivors of critical illness with neurologic impairment. J Crit Care. 2013;17(3):1-9. 7. Macht M, Wimbish T, Clark BJ, Benson AB, Burnham EL, Williams A et al. Diagnosis and treatment of post-extubation dysphagia: results from a national survey. J Crit Care. 2012;27(6):578-86. 8. Malandraki GA, Markaki V, Georgopoulos VC, Psychogios L, Nanas S. Postextubation dysphagia in critical patients: a first report from the largest step-down intensive care unit in Greece. Int J Speech Lang Pathol. 2016;25(2):150-6. 9. Partik B, Pokieser P, Schima W, Schober E, Stadler A, Eisenhuber E et al. Videofluoroscopy of swallowing in symptomatic patients who have undergone long-term intubation. AJR. 2000;174(5):1409-12. 10. Regan J, Walshe M, Lavan S, Horan E, Gillivan-Murphy P, Healy A et al. Postextubation dysphagia and dysphonia amongst adults with COVID-19 in the Republic of Ireland: a prospective multi-site observational cohort study. Clin Otolaryngol. 2021;46(6):1290-9. 11. Skoretz SA, Yau TM, Ivanov J, Granton JT, Martino R. Dysphagia and associated risk factors following extubation in cardiovascular surgical patients. Dysphagia. 2014;29(6):647-54.

Publication Dates

  • Publication in this collection
    25 Nov 2022
  • Date of issue
    2022

History

  • Received
    21 Oct 2022
  • Accepted
    16 Nov 2022
location_on
ABRAMO Associação Brasileira de Motricidade Orofacial Rua Uruguaiana, 516, Cep 13026-001 Campinas SP Brasil, Tel.: +55 19 3254-0342 - São Paulo - SP - Brazil
E-mail: revistacefac@cefac.br
rss_feed Acompanhe os números deste periódico no seu leitor de RSS
Acessibilidade / Reportar erro