Auditory-perceptual training for the assessment of hypernasality in cleft palate speakers: an integrative literature review

ABSTRACT

Purpose:  to describe the auditory-perceptual training for the assessment of hypernasality in individuals with cleft lip and palate.

Methods:  an integrative literature review in the databases Virtual Health Library, SciELO, and PubMed, aimed to answer the following guiding question: 1) What are the characteristics of auditory-perceptual training to assess hypernasality in individuals with cleft lip and palate? Articles in Portuguese and English, available in full access, without the restriction of the publication date, which presented programs of training for speech hypernasality, unprecedented, adapted, or replicated, were included. The pursuit of descriptors, selection, extraction, and synthesis of data was performed by three independent evaluators.

Literature Review:  10 articles were included in this study, based on established criteria. Five articles investigated the effectiveness of training on speech analysis by listeners, regardless of experience level. Another five articles pertained to training when validating speech assessment protocols. Consensus analyses and reference samples were the most used training reported. Perceptual rating of phrases, using the equal appearance scale and in person training, was the most reported one.

Conclusions:  the auditory-perceptual training of listeners to identify hypernasality showed variability in the proposed strategies, particularly when proposed for non-experienced listeners. The difficulty in maintaining acquired skills in the long term is pointed out.

Keywords: Cleft Palate; Velopharyngeal Insufficiency; Speech; Speech Disorders; Mentoring

RESUMO

Objetivo:  descrever as características dos treinamentos perceptivo-auditivos para a avaliação da hipernasalidade em indivíduos com fissura labiopalatina.

Métodos:  revisão integrativa de literatura nas bases de dados Biblioteca Virtual da Saúde, SciELO e PubMed que visou responder a seguinte pergunta norteadora “Quais são as características dos treinamentos perceptivos-auditivos para avaliação da hipernasalidade em indivíduos com fissura labiopalatina?”. Foram incluídos artigos em português e inglês, disponíveis na íntegra, sem restrição de data de publicação, que apresentassem programas de treinamento para hipernasalidade, inéditos, adaptados ou replicados. A busca dos descritores, seleção, extração e síntese dos dados foram feitas por três avaliadores independentes.

Revisão da Literatura:  foram incluídos dez artigos com base nos critérios estabelecidos. Cinco artigos investigaram o efeito do treinamento na análise perceptiva de ouvintes, com ou sem experiência. Outros cinco utilizaram treinamentos de fonoaudiólogos, ao validar protocolos de avaliação da fala. Análises consensuais e amostras de referências foram os treinamentos mais empregados. Julgamentos perceptivos de frases, usando escala de intervalos iguais, em modalidade presencial foram os mais descritos.

Conclusão:  treinamentos perceptivo-auditivos para identificação da hipernasalidade variaram, particularmente, em sua duração e ouvintes incluídos. A dificuldade em manter habilidades adquiridas a longo prazo é apontada.

Descritores: Fissura Palatina; Insuficiência Velofaríngea; Fala; Distúrbios Da Fala; Tutoria

INTRODUCTION

Auditory-perceptual assessment is considered the gold standard in the identification and assessment of speech disorders in individuals with cleft lip and palate and/or velopharyngeal dysfunction. Among these disorders, speech hypernasality is the most frequent one, characterized by an excessive nasal resonance that occurs during the production of oral sounds^1,2. Identifying the presence of hypernasality through auditory-perceptual assessment is essential for the initial diagnosis of velopharyngeal dysfunction and for the assessment of the effectiveness of the treatment, even if subjectively³. Instrumental measures (nasoendoscopy, videofluoroscopy, nasometry, and pressure-flow technique) are commonly used to complement the diagnosis, as they offer valuable information with perceptive findings⁴.

Although essential for clinical purposes, the auditory-perceptual assessment of the speech is subjective and, therefore, can be influenced by internal factors (listeners), external factors (tasks), and the interaction of these factors⁵. The factors related to listeners include the evaluator's degree of experience, habits and errors, perceptive sensitivity, fatigue, lapses, and attention deficits⁵. Speech therapists with experience in evaluating the speech of individuals with cleft lip and palate are preferable for clinical, research, and auditing purposes⁶, even an isolated experience does not guarantee sufficient levels of intra- and inter-rater agreement. Evaluators' internal standards can be unstable over time even for an experienced evaluator⁷.

The factors external to the evaluator include, for example, the type and extent of the sample being analyzed⁸, the phonetic context^8,9), and the presence of coexisting speech disorders, such as compensatory articulations¹⁰ and dysphonia¹¹. Perceptual assessment of a speech aspect can be affected by concurrent changes in other speech subsystems. Scholars point out that listening to several dimensions in the speech signal at the same time can be a difficult task¹² and, therefore, the possible effect of the coexistence of articulatory disorders¹⁰ and dysphonia¹¹ in the auditory-perceptual analysis of hypernasality¹⁰ must be considered.

Considering that the variability in the auditory-perceptual assessment may be due to variations in the listeners' perception and also in the assessment tasks¹³, achieving a high-reliability index can be difficult, even for experienced evaluators^14,15. To minimize errors and biases in the auditory-perceptual assessment and to increase the reliability of this type of assessment, scholars search for strategies that optimize the auditory-perceptual assessment of speech, including speech hypernasality.

According to some scholars¹³, the reliability of the analyses concerns the extension of a method used for the assessment, which offers the same result when measurements are taken repeatedly. As summarized by these scholars¹³, reliability can be achieved through strategies that include speech recordings in audio and/or video with quality equipment for analysis by multiple evaluators; definition of terms, standardization of materials, use of reference samples, and particularly, auditory-perceptual training.

In previous studies, training has been shown to increase the reliability of perceptual analyses by listeners without¹⁶ and with experience in the assessment of speech hypernasality¹⁴. Some authors also suggest that inter-rater agreement rates may increase after a systematic training program for the rater¹⁷. Others pointed out improvement in the concordance indices after training but with no significant difference between the control (without training) and experimental (with training) groups¹⁸.

The importance of describing and carefully analyzing the auditory-perceptual training programs used in research is emphasized in the literature, since this analysis can offer subsidies to improve the training of listeners, both for clinical use and in research¹⁹. In this sense, it is necessary to expand knowledge about auditory-perceptual training for speech hypernasality and, above all, to analyze the training offered in terms of training programs and, particularly, the characteristics of this training (types, duration, and modality of training, speech stimuli used for judgments, scales used to assess hypernasality, and trained listeners). Thus, this study aimed to describe the characteristics of auditory-perceptual training for the assessment of speech hypernasality in individuals presented with cleft lip and palate.

METHODS

This study is an integrative literature review that aims to contribute to the knowledge of the characteristics of auditory-perceptual training offered to favor auditory-perceptual analysis of speech hypernasality. The question was built based on the acronym P - population, C - concept, and C - context. Thus, in this research, P was the individuals with cleft lip and palate or velopharyngeal insufficiency with speech hypernasality, C was the auditory-perceptual training programs, and C was the listeners and other training characteristics, as reported in the methodology. The study aims to answer the following guiding question: 1) What are the characteristics of auditory-perceptual training to assess hypernasality in individuals with cleft lip and palate? The selection of studies contemplating auditory-perceptual training for the assessment of speech hypernasality was carried out through a search in the national and international scientific literature contemplating auditory-perceptual training for the assessment of speech hypernasality in specialized journals available in three databases: Biblioteca Virtual da Saúde (BVS), Scientific Electronic Library Online (SCIELO), PubMed and, later, Google Scholar.

The health descriptors (DeCS) and the corresponding Medical Subject Headings (MeSH) used in the search were fissura palatina (cleft palate), insuficiência velofaríngea (velopharyngeal insufficiency), fala (speech), distúrbios da fala (speech disorders), educação (education), julgamento (judgment) and treinamento (training). The keyword treinamento auditivo (listener training) was used to help in searches. The identification of these descriptors and keywords was carried out by three researchers, independently, from November 2021 to June 2022.

Afterward, a search in the literature was performed using a combination of descriptors. There was no restriction on period, language, or nationality. The Boolean operators “e” (Portuguese) “AND” (English) and “ou” (Portuguese) “OR” (English) were used. The combinations used between the descriptors were fissura palatina AND distúrbios da fala AND fala AND julgamento; education AND speech disorders AND (velopharyngeal insufficiency OR cleft palate) AND listener training; cleft palate AND speech disorders AND judgment; education AND speech disorders AND velopharyngeal insufficiency AND cleft palate AND training; and speech disorders AND cleft palate OR velopharyngeal insufficiency AND training, as shown in Chart 1.

Chart 1
Search strategy for identifying texts

Database	Descriptors	Search strategy	Results
SCIELO	MeSH terms	(((*cleft palate) OR (velopharyngeal insufficiency)) AND (speech disorders)) AND (judgment)	3
DeCS		(((*fissura palatina) OR (insuficiência velofaríngea)) AND (distúrbios da fala)) AND (julgamento)	6
BVS/VHL	DeCS	(fissura palatina) AND (distúrbios da fala) AND (fala) AND (julgamento)	14
PUBMED	MeSH terms	(((("Education" [Subheading])) AND ("Speech Disorders"[Mesh])) AND ("Velopharyngeal Insufficiency"[Mesh])) OR ("Cleft Palate"[Mesh]) AND (Listener Training)	37
		“Speech Disorders”[Mesh]) AND “Cleft Palate”[Mesh]) OR “Velopharyngeal Insufficiency”[Mesh]) AND (Training)	198

• Captions: DeCS, Health Sciences Descriptors; MeSH, Medical Subject Headings.

An article found through a search on the Google Scholar platform, gray literature, was also included, as it met the established criteria, through the descriptors e-learning AND perceptual assessment.

Selection criteria

Original articles in the searched databases and available in full, without any restriction of publication dates, that presented training programs for speech hypernasality and were unpublished, adapted, or replicated, were included. Repeated articles in the databases, theses, or publications that were not in Portuguese or English, as well as studies with another proposal on the topic of interest or involving other speech disorders other than hypernasality, studies in which participants were only instructed to perform the experimental task, without the description of the training program and studies without training details, were excluded.

Data Extraction and Analysis

The search for material in the databases was carried out manually and independently by three evaluators. The selection of material was carried out in three stages: 1) reading the titles of the articles found, 2) reading the abstracts, and 3) reading the articles in full. Initially, the evaluators individually read the titles and abstracts to analyze adherence to the theme. Studies that did not meet the eligibility criteria defined in this study (non-adherence to the theme, duplicate studies, language other than Portuguese or English) were excluded. The three evaluators had a consensus regarding these exclusions. Afterward, the evaluators read the texts in full, individually. The results of each of the evaluators were discussed and a consensus was reached regarding the characteristics of the research.

After the selection of the articles, the following data were extracted from the studies: author and year of publication, study objectives, training methods (type, duration, and modality), speech stimuli used in judgments, scales used to assess hypernasality, characterization of trained listeners and results obtained. Data extraction and organization were performed in an Excel spreadsheet. For extraction, the instrument was adapted, based on previous studies. The items considered for analysis were study identification and training characteristics (types, duration, training modality, speech stimuli and scales used, and trained listeners). Also, a summary of the findings was presented, however, it was not an object of analysis to identify the best perceptual training proposal that would lead to more reliable results.

LITERATURE REVIEW

The search strategy found a total of 259 studies. Of this total, nine articles were found in the SCIELO database, using descriptors in Portuguese and English, three of which were duplicates. After checking the titles, a single study was selected for reading the abstract and, later, this study was read in full, as it dealt specifically with training evaluators to classify hypernasality, with a detailed description of the proposed training.

Another 14 articles, one of which was duplicated, were found by searching the BVS/VHL database, using only descriptors in Portuguese. After reading the titles, two articles were excluded because they had already been included through the search in the SCIELO database, and the other 11 articles did not meet the proposed inclusion criteria. Therefore, no article in this database was selected for the study.

In the PUBMED database, 235 articles were found, using two combinations of descriptors. After the title analysis, the following exclusions were made: 22 of them did not meet the language criteria, one referred to the thesis, 12 articles were not found in full and 25 were duplicated or repeated searches carried out in other databases. With that, 175 articles were analyzed and of these, 117 were excluded due to non-adherence to the theme proposed for this study. Afterward, 58 articles were selected for reading the abstracts, 29 of them were excluded, as they did not present information regarding training for the classification of hypernasality. Finally, 29 articles from the PUBMED database were read in full, and in which 8 of them were included in this study.

A single article, which had not been found in the previous databases, but which was identified in Google Scholar (gray literature) was included due to its importance regarding the proposed theme. Therefore, ten articles are the total sample for this review (Scielo N=1, PUBMED N=8; Google Scholar N=1). Figure 1 shows the article selection process, considering the three databases consulted together.

Figure 1
Flowchart of the article selection process

The objective of this study was to describe the characteristics of auditory-perceptual training for the assessment of speech hypernasality in individuals with cleft lip and palate and to present the results of this training. From the analysis of data extracted from all selected articles, it was observed that most of the scientific productions that address the proposed theme were available in the last 8 years.

This integrative literature review shows the information on aspects related to auditory-perceptual training for the assessment of hypernasality, including (a) the identification of the study involving training; (b) the characteristics of auditory-perceptual training and evaluators included in the training, and (c) the summary of results obtained by training, as found in the database (Chart 2).

Chart 2
Studies contemplating characteristics of auditory-perceptual training for the assessment of speech hypernasality

Author/Year	Study identification (Objective)	Characteristics of the Training			>Results
		Proposal, duration, and modality	Speech stimulus/Scale	Listeners
John et al. (2006)21	Perceptual assessment protocol with training to use the protocol. (To develop an assessment tool (CAPS-A) to use in inter-center audit studies in cleft lip and palate and test the reliability and validity of this assessment tool)	Training offered to evaluators by experienced speech-language therapists in the use of the new assessment protocol (CAPS-A) with consensual analyses (phase 3 of the study) Duration: 6 hours Modality: face-to-face	Counting, sentence repetition, and spontaneous conversation Scale: Equal intervals (5 points)	Speech-language therapists with experience in speech assessment of cleft lip and palate (N=4)	Intra- and inter-rater reliability rates are considered good/very good. The data suggest that after 6 hours of structured training, inter-rater agreement can be achieved. Evaluators considered the CAPS-A protocol to be acceptable and easy to use with appropriate training.
Sell L et al. (2009) 22	Perceptual assessment protocol with training to use the protocol. (To delineate, execute, and evaluate a training program for speech therapists with systematization and reliability in the use of the CAPS-A assessment protocol)	Training offered by CAPS-A speech-language therapists for the use of the protocol. Presentation of the construction and specific aspects of the assessment protocol (protocol development process, review of definitions adopted by the protocol, standardization of numerical scales of speech parameters, review of types of errors in the production of consonants in speech). Offering examples to illustrate the scales used, and exercises to establish consensus. Afterward, practice with consensus (one case) followed by individual analyses, immediately after training and one month later. Duration: 4 days (including two days of specific training followed by consensual practice before individual analyses) Modality: face-to-face	Counting, phrases, rhymes, and spontaneous sampling Scale: Equal intervals (4 points)	Speech-language therapists (N=36)	In general, there was an increase in the reliability of intra- and inter-rater analyses, including the hypernasality speech parameter.
Lee, Whitehill, Ciocca (2009) 16	Training (To investigate the effect of training and feedback on the intra- and inter-rater reliability of the hypernasality judgment)	Calibration (G1, G2, G3) (Focus on hypernasality and coexisting speech disorders G1: Exposure to speech samples, exemplifying hypernasality, nasal emission, vocal disorders, and articulatory errors G2 (practice without feedback) G3 (practice with feedback) Practice: training with a hierarchy of difficulty with 4 steps: 1. identification of the presence or absence of hypernasality, nasal emission, vocal disorders, and articulatory errors 2. Identification of types of disorders 3. Hypernasality degrees discrimination (Identification of the most hypernasal sample between two samples 4. Judgment of hypernasality Duration: - calibration, 30 minutes; - practice 1 hour/group Modality: face-to-face	Phrases Direct Magnitude Scale	Speech-Language Therapy Students (N=36)	A significant difference in inter-rater agreement between the groups with practice (training with and without feedback) and the group without practice (exhibition of samples). Between the two groups that performed the practice, there was no significant difference. Both groups with practice showed an increase in the reliability of intra-rater and inter-rater analyses. Therefore, training (practice with and without feedback) helped improve the reliability of hypernasality ratings.
Chapman et al. (2016) 17	Perceptual assessment protocols with training to use the protocol. (To describe the reliability indices of the speech assessment of two perceptual assessment protocols CAPS-A and CAPS-A-AM) *CAPS-A-AM: modifications made to the CAPS-A protocol	Training offered using the CAPS-A (Study 1) Training included a description of the steps and details of the procedures adopted; phonetic transcription and classification of speech parameters using scales, and practices with consensus. Duration: 3 days Modality: face-to-face Individual analyses Study 1 (CAPS-A): before, immediately, one month after training Study 2 (CAPS-A-AM): analyses on two moments, 5 weeks after study 1	Counting, rhymes, and phrases Scale: Equal intervals (5 points)	Speech-language therapists with experience in speech assessment of cleft lip and palate (N=9) Study 1: N=9 Study 2: N=6, (also included in Study 1)	Study 1 CAPS-A: Significant improvements in inter-rater reliability indices after training, including hypernasality parameter. Study 2 CAPS-A-AM: There were no significant differences in results between raters, including for hypernasality. Inter-rater reliability indices of Study 2 were lower than those of Study 1. After adjusting the data analysis procedures, the findings were similar. In both studies, there was good intra-rater reliability.
Butts et al. (2016) 18	Training (To assess the ability of Otorhinolaryngology (ENT) residents to assess hypernasality in patients with velopharyngeal dysfunction)	Educational Module (Speech samples, explanations about velopharyngeal dysfunction concerning clinical cases) and questionnaire about the module). Educational module offered only for the experimental group, between the classifications of the speech samples, initial and final, performed by the two groups (control and experimental), with one to two weeks of interval between the classifications. Duration: 40 minutes Modality: online	Speech samples (module) not informed. Scale: Equal intervals (4 points)	ORL Residents (N=30)	Improvement in the percentage of agreement (experimental group) after training, but without statistical difference concerning the control group. Greater accuracy in classifications for absent and severe grades
Oliveira, et al. (2016) 14	Training (To investigate the influence of previous training on the agreement between different raters in the perceptual judgment of hypernasality)	Definition of criteria and establishment of samples of reference by experienced speech therapists (consensus) Duration: unspecified Modality: face-to-face	Phrases and counting Scale: Equal intervals (4 points)	Speech-language therapists with experience in speech assessment of cleft lip and palate (N=3)	Increased intra- and inter-rater agreement rates
Spruijt et al. (2018) 23	Perceptual assessment protocols with training to use the protocol. (To measure intra- and inter-rater reliability using Dutch Cleft Speech Evaluation Test (DCSET) and convert DCSET into universal scales)	Training for the use of modifications in the Dutch Cleft Speech Evaluation Test (DCSET) based on video reviews and consensus analysis) (Phase 2 of the study) Review of scales for analysis of resonance and speech production errors, after training (Phase 2). Phase 2 was performed after phase 1 (protocol without reviews) Duration: Unknown Modality: face-to-face	Phrases Scale: Equal intervals (3 points) for hypernasality	Speech-language therapists with experience in speech assessment of cleft lip and palate (N=2)	Phase 1: did not show the expected results; adjustments were made regarding the use of the scale of some speech parameters. Level 2: reliability scores ranged from average to good (all speech parameters) Hypernasality: Reliability of analyses between evaluators (oral samples): Average
Bruneel et al. (2020) 24	Perceptual assessment protocols with training to use the protocol. (To develop and validate an instrument in the Belgian language for perceptual assessment in patients with cleft palate)	Phase 1: Preliminary study Introduction of speech variables and description of the structure of the assessment protocol, with a reference sample for each degree and type of speech disorder (two hours) Consensus practice (1 hour and a half) Total duration: 3 and a half hours Modality: face-to-face Phase 2 (validation): Description and explanation of the adopted definitions and parameters, description of the structure of the assessment protocol, and classification scales. Consensus practice. Total duration: 4 hours	Counting, phrases, and connected speech Scale: Equal intervals (4 points)	Speech-language therapists, (N=2; study 1) Speech-language therapists (N=4; study 2)	Phase 1: In general, indices with good inter-rater analysis reliability, including hypernasality. Phase 2: In general, good intra- and inter-rater reliability, including speech hypernasality. In both phases, inter-rater reliability was lower than intra-rater reliability.
Lohmander et al. (2021)13	Training (To evaluate the training result, short and long-term, through an e-learning tool)	Training using a platform (PUMA website) with clinical cases and feedback from speech therapists experienced in the assessment of speech disorders in cleft lip and palate. Training is offered during a structured course (teaching activity, including lectures, seminars, and laboratory activities), in the student’s free time, through the website. Two groups of students from two different universities were involved in the study. Perceptual training included: - Listening to speech samples containing examples of different types and degrees of speech disorders in individuals with cleft lip and palate - Phonetically transcribing a video sample, with the possibility of comparison with analysis by experienced speech-language therapists - Analyzing speech samples, using a scale of equal intervals, with the possibility of comparison with audio samples with consensual analyses of speech-language therapists Training carried out individually or jointly. Number of sessions and time devoted to training: unspecified. Modality: online Training performed in week 2. Week1: instruction/ pre-test; Week 3: post-test	Isolated words Scale: Equal intervals (4 points)	Speech-Language Therapy Students (N=45)	A total of 16.5 hours (transcription activities) and 8.5 (analyses with scales) were used for training by the total number of students. There was a significant improvement in phonetic transcription, after training, for total students. A significantly higher number of responses agree with the assessment of experienced speech-language therapists, after training, for the hyponasality and weak intraoral pressure variables for the total number of students. An improvement trend in responses to hypernasality was observed for one of the groups of students. Positive comments from students regarding accessibility and practice time. The e-learning indicated an improvement in the ability of the students (evaluators without experience) in the auditory-perceptual assessment.
Bruneel et al. (2022) 20	Training (To assess the immediate and long-term effect of perceptual training on the reliability of intra- and inter-rater analyses)	Definition of criteria and presentation of audio and audiovisual speech samples for practice in judging speech parameters (hypernasality, hyponasality, nasal emission, and nasal turbulence), in addition to training for assessing speech intelligibility and acceptability. Additional samples for training the judgment of hypernasality, hyponasality, nasal emission, and nasal turbulence, first individually followed by group discussion (consensus judgments). Training speech samples differed from test samples (pre-training, short-term post-training, and long-term post-training - 1 month). Duration: 2 Hours Modality: face-to-face	Spontaneous samples and phrases Scale: Equal intervals 5 points	Speech-Language Therapy Students (N=31)	In general, the training positively affected the reliability results of the analyses and the students’ confidence in making the judgments (another aspect of interest in the study); however, these findings were dependent on the analyzed speech variable and at the time of the measurement. Little or no training effect (short and long-term) was observed for hypernasality, air emission, and nasal turbulence variables.

• Captions: N = Number of participants; ENT = Otorhinolaryngology

Regarding the identification of studies, five of the ten reviewed studies had the main objective of describing perceptual training and verifying the effect of training on the reliability of listeners' analyses of the speech parameters, including hypernasality^{13,14,16,18,20}. These studies were named “training” in this review. One of these five articles was national¹⁴ and four were international^13,16,18,20. The other five studies aimed to develop and validate protocols for perceptive assessment of the speech of individuals with cleft lip and palate and proposed training to verify the agreement between different listeners when using the developed protocols^17,21-24). These studies were named “perceptual assessment protocol with training for protocol use” in this review. All of these five articles were international.

The study developed in 2009¹⁶ was the forerunner among the studies whose focus was to describe auditory-perceptual training for hypernasality and to verify its outcomes (training)^{13,14,16,18,20}. This study investigated the effect of training and feedback on the intra- and inter-rater reliability of the hypernasality judgment in non-experienced listeners. Two other studies carried out in 2016 aimed to assess the ability of Otorhinolaryngology residents to classify hypernasality in patients with velopharyngeal dysfunction¹⁸ and to investigate the influence of experienced evaluators' training on the agreement in the perceptive judgment of hypernasality before and after the prior training¹⁴.

Two other more recent investigations were carried out, one in 2021¹³ and the other in 2022²⁰. The study conducted in 2021¹³ aimed to evaluate the results of training (short and long-term) through an e-learning tool, developed to evaluate speech characteristics related to cleft lip and palate and the students' perception of the training proposal presented. The study conducted in 2022²⁰ aimed to evaluate the immediate and long-term effect of perceptual training on the reliability of intra- and inter-rater analyses, in addition to the student's experience in auditory-perceptual assessments of speech in patients with cleft palate, using an assessment protocol developed in a study previous²⁴.

Based on the objectives of these five studies, it was verified that there is a constant and even recent concern of scholars in proposing auditory-perceptual training that favors auditory-perceptual analysis of speech hypernasality. This concern has even led scholars to propose auditory-perceptual training for the assessment of speech hypernasality directed at speech therapists already experienced in assessing the speech of individuals with cleft lip and palate and/or velopharyngeal dysfunction (VPD)¹⁴. Scholars are also concerned about verifying the effect of long-term training, particularly when it is offered to untrained listeners^13,20. In addition, there is a tendency to seek information about the students' perception of the presented proposal²⁰.

Regarding the studies that used auditory-perceptual training of aspects of speech in cleft lip and palate, when developing and validating speech assessment protocols for individuals with cleft lip and palate, the study developed in 2006²¹ was the forerunner. This study aimed to develop an assessment instrument - Cleft audit protocol for speech (CAPS-A) to use in inter-center audit studies in cleft lip and palate and to test the reliability and validity of this assessment instrument. Later, in 2009, scholars²² aimed to design, execute, and evaluate a training program for speech therapists with systematization and reliability in the use of the CAPS-A assessment protocol, directing the problems of sample standardization, data acquisition, recording, reproduction, and listening guidelines.

In 2016, researchers¹⁷ sought to describe the speech assessment reliability indices of two perceptual assessment protocols (Cleft Audit Protocol for Speech-Augmented - CAPS-A and Cleft Audit Protocol for Speech-Augmented-Americleft Modification - CAPS-A-AM) developed to assess speech outcomes in inter-center collaborative studies and investigated the effect of training on agreement between different listeners. For this, two studies were conducted, one using CAPS-A and the other with modifications in this assessment protocol (CAPS-A-AM).

A study developed in 2018²³ aimed to measure inter- and intra-rater reliability using the Dutch Cleft Speech Evaluation Test (DCSET) and convert the DCSET into universal scales. In 2020, a study aimed to develop and validate an instrument in the Belgian language for perceptual assessment in patients with cleft palate²⁴, based on a previous assessment protocol (CAPS-A)21. Intra- and inter-rater reliability indices were reported (including for hypernasality) after a 4-hour training that included protocol presentation and consensus practices.

Based on the objectives proposed in these five studies, it appears that auditory-perceptual training is considered an important strategy to obtain the reliability of the auditory-perceptual analysis of hypernasality, when developing and validating protocols for evaluating the speech of individuals with cleft lip and palate^{17), (21-24}.

Characteristics of auditory-perceptual training for the assessment of hypernasality

Types of training

The types of auditory-perceptual training to assess speech hypernasality varied among the five studies that aimed to describe perceptual training and verify its outcomes in the hypernasality speech parameter (training). A single study¹⁶ used practice (with and without feedback) carried out in stages, with a gradation of levels of difficulty (four levels, from easiest to hardest) to prepare two groups of untrained listeners to classify degrees of hypernasality¹⁶. The other group included in the study did not receive training and was only exposed to speech samples (passive listening).

Two studies^14,20 used consensus analyses. One of these studies¹⁴ used the definition of criteria, followed by the establishment of reference samples, consensually established by speech-language therapists with experience in evaluating the speech of individuals with cleft lip and palate for later use of these references in the judgment of hypernasality. In this study, therefore, consensual analyses were essential for establishing references that were used by speech-language therapists in their analyses of speech samples with different degrees of hypernasality. The other study involving consensus analyses²⁰ used criteria definition, followed by the presentation of audio and audiovisual speech samples for practice in the judgment of speech parameters (including hypernasality), with additional samples for training in the judgment of these parameters, first individually and afterward, consensually. The discussion of analyses by listeners (untrained) was pointed out in this study as an important strategy that can favor the learning of these listeners.

One study¹³ used reference samples (anchors) to favor the perceptive analyses of non-experienced listeners. In this study, an e-learning platform was used for perceptual assessment focusing on speech disorders in cleft lip and palate (including hypernasality). On this platform, basic information about the condition or subcategories of interest for analysis is offered, followed by information and exercises (including videos and audio) referring to the different domains to be evaluated. In the study, the perceptual training was carried out through the platform with access to video (for phonetic transcription) and audio, contemplating examples of types and degrees of speech disorders (including hypernasality), with the possibility of comparison with samples established consensually by experienced speech therapists (reference samples), aiming to familiarize listeners (students from two universities) with the types of errors, in addition to calibrating them and, in one instance, enabling them to perform reliable analyses. Another study¹⁴ also used reference samples, but these references were pre-established by consensus analyses, as already mentioned. In general, auditory-perceptual training, using fixed external references (pre-established) is considered essential to reduce the impact of internal factors related to the evaluator (for example, experience) and, also, factors related to the task imposed in the evaluation^16,17,22.

On study¹⁸ used the exposure of listeners to speech samples (passive listening). This study proposed access to an educational module contemplating speech sample, in addition to explanations about aspects of velopharyngeal dysfunction and clinical correlation. However, no further details were reported on the training used to classify hypernasality.

The types of auditory-perceptual training to assess hypernasality used in the five studies focused on the development, measurement, and validation of speech assessment protocols were well-defined and not very variable. In general, these studies^17,21-24 described carrying out practical exercises to obtain consensual analyses, as part of the training carried out.

More specifically, a study¹⁷ proposed a workshop to carry out the training, with a presentation of the materials and methodology of the CAPS-A protocol. All parameters and definitions were described to the participants, including categories and subcategories, use and standardization of scales. As part of the training, participants performed tasks that included the classification of speech and transcription parameters, as well as practical exercises to obtain consensus. After the training, changes were proposed in some speech samples, a presentation of videos instead of audio, in addition to subtle adjustments in speech parameters and definitions of these parameters (CAPS-A-AM). In another study²¹ the training was established by consensus and involved the presentation of six cases, and included the definition of criteria, being offered to the speech-language therapists who participated in the training, qualitative descriptions regarding the use of the instrument before performing the task.

Another study²² proposed training led by experienced professionals familiar with the CAPS-A assessment protocol. Initially, information on the development process of the CAPS-A assessment protocol was presented for auditing purposes, with a review of the definitions adopted by the protocol (use and standardization of numerical scales of speech parameters, including hypernasality) and a review of the types of errors in the production of consonants in speech of individuals with cleft lip and palate. For this, videos and audio were presented to illustrate the aspects of speech to be evaluated and cases for consensus analysis. After, it was requested that each speech-language therapist analyze auditory or visually the material offered, followed by a discussion of the findings jointly to obtain consensus.

One study²³ used the DCSET protocol to evaluate the speech parameters of children with cleft lip and palate, using audio recordings. After that, modifications in this assessment protocol were proposed from consensual discussions (training) in the scales used to evaluate the speech parameters. These discussions were conducted using video recordings of children with cleft lip and palate.

One study²⁴ developed and validated a Belgian protocol based on CAPS-A. In phase 1 (preliminary study), adaptations of some speech parameters and protocol structure were made for perceptual analysis. Reference samples were also presented pre-classified for each speech variable and each degree of the scale (training). Consensus practice was performed for 1 hour and a half. After an experimental session at this stage, the protocol was optimized. Phase 2 (protocol validation) included a description and explanation of definitions, scales for the classification of speech parameters, and a presentation of the structured assessment protocol. Edited speech samples were used to illustrate the corresponding speech variables and scale degrees (training).

The analysis of the ten studies included in the review shows that consensual analyses are the most used trainings^{14,17,20,21-24}, followed using reference samples¹³. A study¹⁴ established reference samples based on consensual analyses, for later use of these references in individual analyses, suggesting the use of consensual analyses and reference samples to favor the analyses of speech-language therapists (listeners with experience).

Consensus analysis requires a group of listeners to actively listen to audio and/or video speech samples. Discussions among evaluators are expected to reach a consensus analysis of the evaluated aspect¹⁹. According to scholars¹⁹, when establishing consensus, it is sought to offer multiple opportunities for listeners to analyze the parameter of speech of interest (hypernasality), aiming at the development of the listener’s ability to accurately quantify the presented speech parameter.

The reference samples are samples pre-classified by experienced speech-language therapists for the listeners to use during their training, and these should compare the new sample to that pre-classified, judging this new sample as more or less hypernasal concerning the external reference¹⁹. The references are considered effective strategies to establish internal standards for the evaluator because they enable them to experience and become familiar with the references used in the training. Therefore, listeners tend to store these models in their memory as internal patterns, that is, the representations are stored in memory as examples²⁵. Some authors¹⁴ argue that the use of anchor samples (reference) during the classification task may result in a significant improvement in the accuracy of speech sample severity classifications of gravity of speech samples, even for experienced listeners.

The practice (with and without feedback), although little explored among the studies that propose perceptual-auditory training for the evaluation of hypernasality of speech, also offers multiple opportunities for listeners to analyze the parameter of speech of interest (hypernasality), for the development of the listener’s ability to accurately quantify the speech parameter presented¹⁹.

Duration of training

The duration of training described in the reviewed articles varied and was dependent on the purpose of the study and the type of training offered. Studies that aimed to describe perceptual training and verify the effect of training on the reliability of the listeners' analyses about hypernasality (training) indicated a duration ranging from 40 minutes (minimum)¹⁸ to two hours (maximum)²⁰. In one study¹³, the duration of individual training was not measured since listeners (students in training) performed the training according to their own time. One study¹⁴ did not show the duration of training performed. Studies that focused on the development, measurement, and validation of speech assessment protocols for individuals with cleft lip and palate pointed to a duration of 2 hours²⁴, 6 hours²¹, 3 days¹⁷, or 4 days²². One study²³ did not inform the duration of the training performed. The analysis of the duration of auditory-perceptual training to assess hypernasality was longer in studies involving the development and validation of speech protocols and shorter in studies involving training, particularly aimed at non-experienced listeners.

Modality of training

Eight of the ten articles reviewed conducted auditory-perceptual training in person^{14,16,17,20-24}. The two remaining studies reported the use of remote platforms to perform perceptual training^13,18. These two studies point to the possibility of auditory-perceptual training of hypernasality and other speech parameters of individuals with cleft lip and palate in the online modality. According to scholars²⁰, this type of training can be advantageous since it enables the participation of students in distance training and also allows listeners to carry out practices in their time. On the other hand, technical problems were pointed out by students who participated in interactive activities (online) using an e-learning platform¹³. By using online learning platforms, it becomes possible to propose continuous perceptual training, in addition to calibrating listeners for studies focused on the outcome of treatment results for the management of speech disorders in cleft lip and palate, which makes this resource very attractive¹³.

Speech stimuli

The speech stimuli used in the ten studies reviewed included: number counting (1 to 10^14,24, 1 to 20^17,21,22,24, 60 to 70^17,22,24), set of phrases^{14,16-18,20-24}, rhymes^17,22, single words¹³, linked and spontaneous word²². For the linked speech stimuli, participants were asked to say the days of the week²⁴.

Six of the ten articles analyzed^{14,17,20-22,24} used combinations of speech samples that included phrases and other stimuli. The following combinations were reported: sentences, counting and spontaneous speak²¹, sentences, counting, spontaneous speak²²; sentences, counting and rhyming¹⁷; sentences and counting¹⁴; sentences, counting and connected speak²² and phrases and spontaneous speak²². One study used only isolated words as a speech stimulus¹³ and three other studies used only phrases^16,18,23. In one study²⁴ the speech stimuli varied according to the age group of the participants. In another study²⁴, children counted from 1 to 10, while adults did from 1 to 20 and 60 to 70. A single article did not provide information about the speech stimuli contained in the recordings used in its study¹⁸.

In general, it was observed that the training offered included combinations of speech stimuli, and phrases were the stimulus most present in training. Sets of oral phrases consisting of controlled stimuli may favor auditory-perceptual analysis of hypernasality⁹ and are commonly used in research and clinical practice.

Scales used in training

Nine of the studies presented in the review used the equal intervals scale, using a score of 3 to 5 points, showing that this type of scale is still the most used in training studies for perceptual analysis of hypernasality. In this type of scale, the evaluator assigns a score to the evaluated aspect, indicating its level of severity, in which the lowest value of the scale refers to the absence of change while the highest value points to the maximum degree of disorder³. Only one of the studies that used the equal interval scale reported using a three-point scale (absent, low hypernasality, and high hypernasality)²³. The others used a scale of 4 or 5 points. A single study of this review used the direct magnitude scale for the perceptual analysis of hypernasality¹⁶.

The use of scales based on proportion (relation), including direct estimation, is advocated by scholars who argue that equal interval scales are inconsistent with the perceptual nature of speech nasality²⁶. According to literature, nasality is a sensation mentally processed as a prosthetic dimension, that is, it differs in terms of changes in quantity or magnitude. Thus, when judging prosthetic stimuli, the listeners do not perceive the intervals between the categories as equal at different points on the scale²⁷. Although scales based on proportion (relation) are pointed out as appropriate for analysis of speech nasality, scales with equal intervals are still the most used clinically and in research, including those that present perceptual training of hypernasality, by favoring comparisons between scales and between evaluators³.

Listeners included in the training.

The evaluators in the review studies included listeners with and without experience in assessing speech hypernasality. Most of the reviewed studies that described perceptual training and verified its outcome in the speech parameter hypernasality (training) included listeners without experience (students in Speech-Language Therapy (N=3)^13,16,20 or residents of Otorhinolaryngology (N=1)¹⁸. A single study¹⁴ proposed auditory-perceptual training for speech-language therapists with experience in the evaluation of speech changes in cleft lip and palate.

Auditory-perceptual training is necessary to reach agreement in auditory-perceptual analyses¹³ and, therefore, represents an important strategy that can be used to prepare listeners for their clinical practices¹⁸. As for inexperienced listeners, Speech-Language Therapy students were selected in three studies^13,16,20, as they represented individuals who should be prepared to conduct clinical assessments of speech aspects, including hypernasality, for initial diagnosis of velopharyngeal dysfunction. Otorhinolaryngology residents were included in a study¹⁸. According to the authors, the assessment of many aspects of speech requires collaboration between speech-language therapists and otorhinolaryngologists; however, the otorhinolaryngologist may be the professional who has the first contact with a symptomatic patient, having the responsibility of carrying out preliminary assessments of speech disorders, including the hypernasality. In this sense, these professionals must be trained to initially identify hypernasality.

A study¹⁴ included experienced listeners (speech-language therapists who work at a craniofacial center). As summarized by some authors¹³, the evaluator's internal standards can be considered unstable, based on previously heard speech samples and, therefore, the evaluator's experience alone does not guarantee intra- and inter-evaluator agreement. Thus, the more exposure to deviant speech, the greater differentiation in internal patterns is achieved, which can result in poor concordance. On the other hand, listeners may begin to converge, in their analyses, when they act together in the same craniofacial center. Furthermore, the coexistence of speech disorders can impact perceptual analyses, including the aspect of hypernasality. In this sense, auditory-perceptual training for experienced speech-language therapists may represent an important strategy to increase reliability levels of hypernasality perceptual analysis.

As expected, all studies focusing on the development, measurement, and validation of speech assessment protocols for individuals with cleft lip and palate included speech-language therapists. These professionals received training to verify whether training would result in increased reliability of responses (intra and between evaluators) for the speech parameters analyzed, including hypernasality. In addition, professionals had the opportunity to maintain the skills developed after training, as part of continuing education²².

Results achieved in studies

In general, the reviewed studies reported favorable results for the use of auditory-perceptual training for the assessment of speech hypernasality. However, the results reported in the studies that described perceptual training and its outcomes (training)^{13,14,16).(18,20} were less expressive than those obtained in the studies that used training, to develop and validate speech assessment protocols for individuals with cleft lip and palate^17,21-14.

One of the five studies that aimed to describe perceptual training and to verify the effect of training on the reliability of the listeners' analysis on hypernasality (training)^{13,14,16,18,20} demonstrated that the training of listeners without experiences (with and without feedback) proposed through a hierarchy of tasks, from the simplest to the most difficult, increased reliability (inter and intra) in the judgment of hypernasality, a fact that did not occur for the group that was only exposed to the samples of speech¹⁶. Similarly, another study¹⁴ showed that, after training, there was an increase in the index of intra and inter-evaluator agreement of the analyses performed, even by listeners with previous experience in the assessment of speech hypernasality. However, three other studies involving non-experienced listeners showed less favorable results^13,18,20.

Particularly, in the study that investigated the ability of Otorhinolaryngology residents to perform the classification of speech hypernasality¹⁸, an improvement in the concordance indices was observed after training, but with no significant difference between the groups (control and experimental, with training). Therefore, this is different from results found in other studies^14,16. In another study¹³, a significant increase in the total performance of Speech-Language Therapy students in phonetic transcription and the classification of some aspects of speech (hyponasality, weak intraoral pressure) was observed with the use of an e-learning instrument for perceptive evaluation of aspects of speech. However, there was only a trend towards improvement in responses for hypernasality and this trend was only observed for one of the two groups of students included in the study. Again, these results differ from those found in other studies^14,16.

Finally, in the study²⁰ that investigated the immediate and long-term effect of training on intra- and inter-rater reliability, the results suggested a positive effect of training on inter-rater reliability, but dependent on the analyzed variable and the time in which the measurement was taken. In general, little or no training effect was observed for the variables hypernasality, air emission, and nasal turbulence, which differed from previous results^14,16.

The analysis of the results obtained in these studies suggests that the variability in the findings involving non-experienced listeners seems to be related to the differences in the training strategies employed, the scales used, and the training modalities offered (online or face-to-face). Some scholars²⁰ argue that training including practices with levels of difficulty¹⁶ may favor the development of the ability to classify speech hypernasality, particularly if this training is offered online¹³, making it possible to include curricular hours for interaction between students and teachers with a transfer of fundamental information. According to these scholars²⁰, training could be optimized by including a comparison of analyses with samples pre-classified by experienced listeners and by exercises to establish consensus analyses among students.

The five studies that used auditory-perceptual training of speech aspects in cleft lip and palate, but with a focus on the development, measurement, and validation of protocols for evaluating the speech of individuals with cleft lip and palate^17,21-24, showed positive results. That is, there were better reliability indices in the hypernasality analyses after speech-language therapists' auditory-perceptual training. Particularly, the precursor study (development of CAPS-A)²¹ suggested that, after a 6-hour training, reliability indexes in perceptual analyses can be achieved. In a later study²², in which a training package was developed for the use of the CAPS-A protocol, the results showed that, in general, there was a significant increase in the reliability indexes for intra- and inter-evaluator analyses for different aspects of speech (including hypernasality). This finding was obtained based on analyses performed immediately after training and repeated after one month of training.

Similar results were pointed out in a later investigation¹⁷, using a training package developed for the use of the CAPS-A protocol¹⁷. Another study²³ observed positive effects arising from training and the standardization of an assessment protocol (Dutch Cleft Speech Evaluation -DCSET). Intra- and inter-rater reliability rates of speech parameters increased after training. The lowest indexes achieved were speech hypernasality. Finally, in the validation process of a study that developed a speech assessment protocol in the Belgian language, the intra- and inter-rater reliability indexes were reported (including for hypernasality) after a 4-hour training that included the presentation of the protocol and consensus practices. The analysis of the results obtained in these studies suggests a positive effect of training on the reliability of the analysis of listeners with experience in the assessment of hypernasality when using training as part of the development or validation of protocols for evaluating the speech of individuals with cleft lip and palate.

This literature review shows that auditory-perceptual training for the assessment of speech hypernasality in individuals with cleft lip and palate is poorly described in the literature, especially for non-experienced listeners. However, there is a tendency to propose perceptual-auditory training for listeners who are students of Speech-language Audiology courses^13,20. This fact reflects the continuous search for structured training aimed at students of Speech-Language Therapy courses to enable them to perform reliable analysis of speech parameters of individuals with cleft lip and palate, including hypernasality²⁰.

In the reviewed studies, some limitations on the training offered were pointed out and may be useful to guide future studies. These limitations include limited time of the auditory-perceptual training offered^20,24, the failure to carry out the pre-training analysis for comparison purposes^17), and the difficulty in achieving a positive effect on the auditory-perceptual classification over time, after auditory-perceptual training of the hypernasality²⁰. Other factors that may affect auditory-perceptual training include restricted speech samples¹⁸ and of evaluators in the trainings^1,16,18, lack of standardization of equipment used in the classifications by the evaluators²¹, the audio quality of the speech samples used^22,23, the use of ordinal scales because they are inconsistent with the perceptive nature (prosthetic dimension) of hypernasality^14,24 and coexistence of other speech disorders when evaluating hypernasality²⁰.

Aiming to meet some of these limitations, scholars reinforce the need to propose, in future studies, structured auditory-perceptual training that allows the maintenance of the skills learned in the long-term²⁰. Recommendations on the need for investigations aimed at understanding the training and the type of feedback that favors the maintenance of the skills developed by listeners in the long-term were made long-term¹⁶ and therefore deserve attention.

Samples of external references established based on consensual analyses (group consensus) or determination of anchor samples by the listeners who will analyze the speech aspects or, even, the combination of them are pointed out as strategies that can favor perceptive analyses of non-experienced speech therapists²⁴ and, therefore, should be considered in future studies. The listeners' skills can also be favored by offering feedback on correct answers in practices involving consensual analysis and by offering additional information such as the presentation of specific cases with precise definitions for the use of the ordinal scale²⁴. The use of proportion scales (direct magnitude estimation) is still suggested in future studies involving training to favor perceptive analyses²⁴.

Perceptual training with an emphasis on speech samples with mild and moderate hypernasality is also recommended to enable more accurate discrimination of these degrees of hypernasality of speech by the listeners¹⁸. The use of simpler perceptual scales is also suggested since they can increase the reliability of classifications and consensus among evaluators²³. In future studies, these aspects can be considered.

It is known that perceptual training, when performed gradually, that is, in stages, with difficulty levels (4 levels, easier to more difficult)¹⁶ favor the analysis of non-experienced listeners and, therefore, should be further explored in future studies. These trainings can also be offered in the online modality¹³. The online modality of auditory-perceptual training is very attractive since it allows the listener to perform perceptual analysis of the evaluated parameter in their own time and an interactive. Therefore, it should be explored in future studies aimed at the auditory-perceptual training of hypernasality.

In future studies, questionnaires should be sent to participants inserted in the online perceptual-auditory training since they can offer valuable suggestions. In a previous study, for example, participants suggested the insertion of technical information in the proposed training sessions to avoid possible problems during the training performance¹³.

This review provides important information on the characteristics of auditory-perceptual training of hypernasality. This information can guide future studies that aim to optimize the perceptual analysis of speech nasality to favor appropriate diagnoses and direct therapeutic processes.

CONCLUSION

Studies involving auditory-perceptual training to identify hypernasality are still scarce. Of those existing, half refer to investigations that aimed to describe and analyze the outcomes of auditory-perceptual training to identify hypernasality. The other half aimed to develop, measure, and validate speech assessment protocols and used auditory-perceptual training for this purpose. The characteristics of speech perceptual training for the assessment of hypernasality varied widely, especially among studies that proposed auditory-perceptual training for inexperienced listeners. The most used types of training were consensual analyses and reference samples. The duration of the training was dependent on the purpose of the study and the type of training used (face-to-face or online). The most recurrent speech stimulus in perceptual training was the set of sentences. Equal interval scales were the most used in the studies. Trained listeners were speech-language therapists, speech-language therapy students, and residents in Otorhinolaryngology. The findings of auditory-perceptual training for the assessment of hypernasality derived from this review point to the need for new training that favors the perceptive analysis of non-experienced listeners and to identify training that can maintain the skills achieved by listeners in the long term.

REFERENCES

• ¹ Zajac DJ, Vallino LD. Speech and resonance characteristics. In: Zajac DJ, Vallino LD. Evaluation and management of cleft lip and palate: a developmental perspective. San Diego: Plural; 2017. p.193-226.
• ² Kummer AW. Management of velopharyngeal insufficiency: the evolution of care and the current state of the art. J. cleft lip palate craniofacial anomalies. 2019;6(2):65-72. https://doi.org/10.4103/jclpca.jclpca_10_19
  » https://doi.org/10.4103/jclpca.jclpca_10_19
• ³ Baylis AL, Munson B, Moller KT. Perceptions of audible nasal emission in speakers with cleft palate: a comparative study of listener judgments. Cleft Palate Craniofac J. 2011;48(4):399-411. http://dx.doi.org/10.1597/09-201 PMID:20572776.
  » http://dx.doi.org/10.1597/09-201
• ⁴ Kummer AW. Speech evaluation for patients with cleft palate. Clin Plast Surg. 2014:41(2):241-51. https://doi.org/10.1016/j.cps.2013.12.004 PMID:24607192.
  » https://doi.org/10.1016/j.cps.2013.12.004
• ⁵ Eadie TL, Baylor CR. The effect of perceptual training on inexperienced listeners' judgments of dysphonic voice. J. Voice. 2006;20(4):527-44. http://dx.doi.org/10.1016/j.jvoice.2005.08.007 PMID:16324823.
  » http://dx.doi.org/10.1016/j.jvoice.2005.08.007
• ⁶ Howard S, Lohmander A. Cleft palate speech: assessment and intervention. Nova Jersey: John Wiley & Sons; 2011.
• ⁷ Keuning KH, Wieneke GH, Dejonckere PH. The intrajudge reliability of the perceptual rating of cleft palate speech before and after pharyngeal flap surgery: the effect of judges and speech samples. Cleft Palate Craniofac J. 1999;36(4):328-33. https://doi.org/10.1597/1545-1569_1999_036_0328_tirotp_2.3.co_2 PMID:10426599.
  » https://doi.org/10.1597/1545-1569_1999_036_0328_tirotp_2.3.co_2
• ⁸ Yamashita RP, Borg E, Granqvist S, Lohmander A. Reliability of hypernasality rating: comparison of 3 different methods for perceptual assessment. Cleft Palate Craniofac J. 2018;55(8):1060-71. https://doi.org/10.1177/1055665618767116 PMID:29634363.
  » https://doi.org/10.1177/1055665618767116
• ⁹ Marino VC de C, Dutka J de CR, Manicardi FT, Gifalli G, Silva PP, Pegoraro-Krook MI. Influence of speech stimuli in the auditory perceptual identification of hypernasality in individuals with cleft lip and palate. CoDAS. 2020;32(6):e20190269. https://doi.org/10.1590/2317-1782/20202019269 PMID:33331425.
  » https://doi.org/10.1590/2317-1782/20202019269
• ¹⁰ Lee A, Potts S, Bressmann T. Speech-language therapy students' auditory-perceptual judgements of simulated concurrent hypernasality and articulation disorders. Clin Linguist Phon. 2020;34(5):479-92. https://doi.org/10.1080/02699206.2019.1655666 PMID:31429313.
  » https://doi.org/10.1080/02699206.2019.1655666
• ¹¹ Imatomi S. Effects of breathy voice source on ratings of hypernasality. Cleft Palate Craniofac J. 2005;42(6):641-8. https://doi.org/10.1597/03-146.1 PMID:16241176.
  » https://doi.org/10.1597/03-146.1
• ¹² Kent RD. Hearing and believing some limits to the auditory-perceptual assessment of speech and voice disorders. Am J Speech Lang Pathol. 1996;5(3):7-23. http://dx.doi.org/10.1044/1058-0360.0503.07
  » http://dx.doi.org/10.1044/1058-0360.0503.07
• ¹³ Lohmander A, Klintö K, Schalling E, Portela AS, Johansson K, McAllister A. Students take charge of learning - using e-learning in perceptual assessment in speech-language pathology. Scand. J. Educ. Res. 2021;65(3):468-80. http://dx.doi.org/10.1080/00313831.2020.1716064
  » http://dx.doi.org/10.1080/00313831.2020.1716064
• ¹⁴ Oliveira ACASF, Scarmagnani RH, Fukushiro AP, Yamashita RP. The influence of listener training on the perceptual assessment of hypernasality. CoDAS. 2016;28(2):141-8. http://dx.doi.org/10.1590/2317-1782/20162015163 PMID:27191877.
  » http://dx.doi.org/10.1590/2317-1782/20162015163
• ¹⁵ Ahl R, Harding-Bell A. Comparing methodologies in a series of speech outcome studies: challenges and lessons learned. Cleft Palate Craniofac J. 2018;55(1):35-44. https://doi.org/10.1177/1055665617718546 PMID:34162055.
  » https://doi.org/10.1177/1055665617718546
• ¹⁶ Lee A, Whitehill TL, Ciocca V. Effect of listener training on perceptual judgement of hypernasality. Clin Linguist Phon. 2009;23(5):319-34. http://dx.doi.org/10.1080/02699200802688596 PMID:19399664.
  » http://dx.doi.org/10.1080/02699200802688596
• ¹⁷ Chapman KL, Baylis A, Trost-Cardamone J, Cordero KN, Dixon A, Dobbelsteyn C et al. The Americleft Speech Project: a training and reliability study. Cleft Palate Craniofac J. 2016;53(1):93-108. http://dx.doi.org/10.1597/14-027 PMID:25531738.
  » http://dx.doi.org/10.1597/14-027
• ¹⁸ Butts SC, Truong A, Forde C, Stefanov DG, Marrinan E. Perceptual assessment of velopharyngeal dysfunction by otolaryngology residents. Otolaryngol Head Neck Surg. 2016;155(6):1034-9. https://doi.org/10.1177/0194599816662247 PMID:27484234.
  » https://doi.org/10.1177/0194599816662247
• ¹⁹ Walden PR, Khayumov J. The use of auditory-perceptual training as a research method: a summary review. J Voice. 2020;36(3):322-34. https://doi.org/10.1016/j.jvoice.2020.06.032 PMID:32747174.
  » https://doi.org/10.1016/j.jvoice.2020.06.032
• ²⁰ Bruneel L, Danhieux A, Van Lierde K. Training speech pathology students in the perceptual evaluation of speech in patients with cleft palate: reliability results and the students' perspective. Int. J. Pediatr. Otorhinolaryngol. 2022;157:111145. https://doi.org/10.1016/j.ijporl.2022.111145 PMID:35468487.
  » https://doi.org/10.1016/j.ijporl.2022.111145
• ²¹ John A, Sell D, Sweeney T, Harding-Bell A, Williams A. The Cleft Audit Protocol for Speech-Augmented: a validated and reliable measure for auditing cleft speech. Cleft Palate Craniofac J. 2006;43(3):272-88. https://doi.org/10.1597/04-141.1 PMID:16681400.
  » https://doi.org/10.1597/04-141.1
• ²² Sell D, John A, Harding-Bell A, Sweeney T, Hegarty F, Freeman J. Cleft Audit Protocol for Speech (CAPS-A): a comprehensive training package for speech analysis. Int J Lang Commun Disord. 2009;44(4):529-48. https://doi.org/10.1080/13682820802196815 PMID:18821108.
  » https://doi.org/10.1080/13682820802196815
• ²³ Spruijt NE, Beenakker M, Verbeek M, Heinze ZCM, Breugem CC, Mink van der Molen AB. Reliability of the Dutch Cleft Speech Evaluation Test and Conversion to the Proposed Universal Scale. J. Craniofac. Surg. 2018;29(2):390-5. https://doi.org/10.1097/SCS.0000000000004261 PMID:29381632.
  » https://doi.org/10.1097/SCS.0000000000004261
• ²⁴ Bruneel L, Alighieri C, D'haeseleer E, Kissel I, Adriaansen A, Sseremba D et al. Reliability results of perceptual ratings of resonance, nasal airflow and speech acceptability in patients with cleft palate by Ugandan speech-language pathologists following a two-day workshop. Int. J. Pediatr. Otorhinolaryngol. 2020;136:110191. https://doi.org/10.1016/j.ijporl.2020.110191 PMID:32593063.
  » https://doi.org/10.1016/j.ijporl.2020.110191
• ²⁵ Chan KM, Yiu EM. A comparison of two perceptual voice evaluation training programs for naive listeners. J Voice. 2006;20(2):229-41. https://doi.org/10.1016/j.jvoice.2005.03.007 PMID:16139475.
  » https://doi.org/10.1016/j.jvoice.2005.03.007
• ²⁶ Baylis AL, Chapman K, Whitehill TR. Validity and reability of visual scaling for assessment of hypernasality and audible nasal emission in children with repaired cleft palate. Cleft Palate Craniofac J. 2015;52(6):660-70. https://doi.org/10.1597/14-040 PMID:25322442.
  » https://doi.org/10.1597/14-040
• ²⁷ Stevens SS. Perceptual magnitude and its measurement. In: Carterette C, Friedman MP, editors. Handbook of perception: psychophysical judgment and measurement. New York: Academic Press; 1974. p.22-40.

• A study conducted at Universidade Estadual Paulista - UNESP, Faculdade de Filosofia e Ciências, Marília, São Paulo, Brazil.
• Financial support: Conselho Nacional de Desenvolvimento Científico e Tecnológico, proposal number 4404.

Publication Dates

• Publication in this collection
  27 Oct 2023
• Date of issue
  2023

History

• Received
  09 Mar 2023
• Accepted
  07 July 2023