Evidence of validity and reliability of the Compressed Speech Test with Figures

ABSTRACT

Purpose

To seek evidence of validity and reliability for the Compressed Speech Test with Figures.

Methods

The study was subdivided into three stages: construct validation, criteria and reliability. All participants were aged between 6:00 and 8:11. For the construct, Compressed Speech with Figures and the gold standard Adapted Compressed Speech test were applied to children with typical phonological development. For criterion analysis, Compressed Speech with Figures was applied in two groups, with typical (G1) and atypical (G2) phonological development. Finally, the application protocols underwent analysis by two Speech Therapists, with experience in the area of Central Auditory Processing, seeking to obtain an inter-evaluator reliability analysis.

Results

The correlation test indicated an almost perfect construct (correlation 0.843 for the right ear and 0.823 for the left ear). In the criterion analysis, it was noticed that both groups presented satisfactory results (G1 = 99.6 to 100%; G2 = 96 to 96.5%). The reliability analysis demonstrated that the protocol is easy to analyze, as both professionals presented unanimous responses.

Conclusion

It was possible to obtain evidence of validity and reliability for the Compressed Speech with Figures instrument. The construct analysis showed that the instrument measures the same variable as the gold standard test, with an almost perfect correlation. In the criterion analysis, both groups presented similar performance, demonstrating that the instrument does not seem to differentiate populations with and without mild phonological disorder. The inter-evaluator reliability analysis demonstrated that the protocol is easy to analyze and score.

Keywords:
Test Validity; Child; Speech; Auditory Perception; Auditory Perception Disorders

RESUMO

Objetivo

Buscar evidências de validade e fidedignidade para o Teste de Fala Comprimida com Figuras.

Método

O estudo foi subdividido em três etapas: validação de construto, critério e fidedignidade. Todos os participantes tinham idade entre 6:00 e 8:11. Para o construto, aplicou-se o Fala Comprimida com Figuras e o teste padrão ouro Fala Comprimida Adaptado em crianças com desenvolvimento fonológico típico. Para análise de critério, aplicou-se o Fala Comprimida com Figuras em dois grupos, com desenvolvimento fonológico típico (G1) e atípico (G2). Por fim, os protocolos de aplicação passaram pela análise de duas Fonoaudiólogas, com experiência na área do Processamento Auditivo Central, buscando obter uma análise de fidedignidade interavaliadores.

Resultados

O teste de correlação indicou um construto quase perfeito (Rho=0,843 para orelha direita e Rho=0,823 para orelha esquerda). Na análise de critério, percebeu-se que ambos os grupos apresentaram resultados satisfatórios (G1 = 99,6 a 100%; G2 = 96 a 96,5%). Já a análise de fidedignidade demonstrou que o protocolo é de fácil análise, pois ambos os profissionais apresentaram respostas unânimes.

Conclusão

Foi possível obter evidências de validade e fidedignidade para o instrumento de Fala Comprimida com Figuras. A análise de construto evidenciou que o instrumento mede a mesma variável que o teste padrão outro, com correlação quase perfeita. Na análise de critério, ambos os grupos apresentaram desempenho semelhante, demonstrando que o instrumento não parece diferenciar populações com e sem transtorno fonológico leve. A análise de fidedignidade interavaliador demonstrou que o protocolo é de fácil análise e pontuação.

Descritores:
Validade dos Testes; Criança; Fala; Percepção Auditiva; Transtornos da Percepção Auditiva

INTRODUCTION

Central Auditory Processing (CAP) refers to a set of specific skills on which the subject depends to understand what they hear(¹). More specifically, CAP is the construction made based on the auditory signal, aiming to make the information useful, and cannot be summarized only in the perception of sound, but also in the identification, location, attention, analysis, memorization and retrieval of information(²).

To assess CAP skills, different behavioral tests are needed, which simulate challenging everyday situations. The complete battery must consist of low-redundancy monaural tests, dichotic tests, temporal processing tests and binaural interaction tests(³).

In low-redundancy monaural tests, the target category of this study, there is an extrinsic reduction of the speech signal, through modification of frequency, time and intensity. However, until then, all standardized tests in this category required verbal responses(⁴), which became an obstacle in the assessment of this auditory ability in children with speech sound disorders, for example, since it was not possible to decipher whether the error came from a lack of sound discrimination or unintelligible speech production. Thus, the need to construct a new test, based on figures, was understood(⁵).

The development of an instrument in any area of health requires immersion in Psychometrics, to analyze whether it really measures what it proposes and whether its answers are reliable(⁶). The international literature still emphasizes that only after validity and reliability studies should the materials be made available for use, whether in clinical or scientific practice(⁷).

Therefore, after the test development stage(⁵), as psychometric studies suggest, the instrument was applied to a pilot sample, demonstrating its easy and quick applicability. Thus, the objective of this study was to continue the validation process of the Compressed Speech with Figures instrument(⁵), seeking evidence of validity criterion, construct and reliability in children with typical phonological development and phonological disorders.

METHODS

Ethical aspects

This study is quantitative and cross-sectional in nature. It followed all ethical precepts in accordance with resolution 510/16 of the National Health Council. Collection began after approval by the Research Ethics Committee, under number 5,197,934, which required all participants to consent to voluntary participation and sign the Free and Informed Consent Form (FICF) or Term of Assent, in which, contained information about the procedures performed, risks, benefits and confidentiality of research data. All assessments were performed at the Audiology Outpatient Clinic of a higher education institution.

Participants and procedures

The total sample consisted of 30 subjects, of both sexes, aged between 6:00 and 8:11, who were recruited by convenience. The age group was selected based on studies that consider phonological disorder up to nine years of age, being called residual speech error after this period(⁸), as well as studies that suggest early investigation of central auditory abilities such as predictive factor for speech development(¹,⁹). The subjects underwent the following procedures:

1. Visual inspection of the external acoustic meatus, using a Mikatos otoscope;

2. Pure Tone Threshold Audiometry and Logoaudiometry, using the Interacoustics brand audiometer, model AD229e;

3. Acoustic Immittance Measurements, using the Interacoustics brand Immittance Meter, model AT235;

4. Orofacial Myofunctional Assessment with Score (OMES)(¹⁰), seeking to identify alterations in the phonoarticulatory organs and the functions of the stomatognathic system that could interfere with the correct production of sounds;

5. Phonological Assessment Instrument (INFONO)(¹¹), seeking to analyze the presence of phonological disorders, through the application of the picture naming stage.

All children should have hearing thresholds within normal limits bilaterally, that is, 15 dB up to seven years of age(¹²) and up to 20 dB for older ages(¹³); Logoaudiometry compatible with audibility thresholds; type A tympanometric curves, that is, the maximum point of compliance occurs between +100 and -100 daPa and the amplitude between 0.3 and 1.6 ml(¹⁴); contralateral acoustic reflexes present at frequencies of 0.5, 1, 2 and 4 kHz bilaterally; normality in the Orofacial Myofunctional Examination(¹⁰). Subjects who presented evident neurological and/or psychological alterations, phonetic disorders, malformations and congenital orofacial syndromes were excluded from the sample. It is worth noting that all assessments were performed in a single day, taking approximately 1 hour and 30 minutes each.

The INFONO result was decisive for the distribution of the sample. Group 1 (G1) was composed of 22 children with typical phonological development, who responded to the Adapted Compressed Speech (ACS)(¹⁵), gold standard, and Compressed Speech with Figures (CSF)(⁵) tests. Group 2 (G2) was made up of eight children with atypical phonological development, who responded only to the CSF test, due to their speech disorder.

Chart 1 presents the description of the participants regarding gender and age.

Chart 1
Description of the sample regarding gender and age

	Typical group (G1)	Atypical group (G2)
N	22	8
Gender M/F	11/11	4/4
Age (average)	7.26	7

• Caption: G1 = Group 1; G2 = Group 2; N = number of subjects; M = male; F = female

The study was performed in three stages: 1. Construct validation; 2. Criterion validation; 3. Reliability. Chart 2 presents a description of the participants and selection criteria for each stage of the research.

Chart 2
Description of participants and selection criteria

Stages	Subjects	Selection criteria
Stage 1. Construct validation	22 typical children	Children aged between 6:00 and 8:11 with typical phonological development.
Stage 2. Criterion validation	22 typical children and 8 atypical children	Children aged between 6:00 and 8:11, with typical and atypical phonological development.
Stage 3. Reliability analysis (inter-evaluator)	02 speech therapists	Speech therapists, with clinical experience in the area of CAP.

The ACS test(¹⁵) consists of the presentation of two lists with 25 two-syllable words, each, monaurally, compressed 60% of the time. The child was instructed to repeat the word heard in an accelerated manner, in the way he/she understood. The CSF(⁵) test, despite having the same structure, that is, two lists with 25 two-syllable words each, with 60% time compression, is supported by visual material so that the child, instead of repeating the words heard, can point to the image that represents it. Therefore, during the application, the child remained with the image booklet in hand so that he could answer the test. Both tests were applied with an intensity of 40 dBSL, monaural presentation, after calibration of the AD229e audiometer, using supra-aural headphones.

As mentioned above, initially, construct analysis was performed based on the responses from G1, through a comparison between ACS and CSF. For this purpose, a Spearman correlation analysis was performed.

Subsequently, criterion analysis was performed by comparing G1 and G2 in the CSF test. A normality analysis was performed for the variables CSF RE (%) and CSF LE (%) using the Kolmogorv-Smirnov test, which rejected the hypothesis (p = 0.001). Therefore, the variables were compared using the non-parametric Wilcoxon test.

Furthermore, after performing the tests, the application protocols were analyzed by two speech therapists, with experience in the CAP area, seeking to obtain an inter-rater reliability analysis. All data were stored and analyzed using SPSS v.22 statistical software, with p ≤ 0.05 being considered significant results.

RESULTS

Table 1 presents a summary of the variables analyzed for construct validity. The variables ACS right ear (RE) and CSF RE were positively and strongly correlated (Rho=0.843; p ≤0.001), as well as ACS left ear (LE) and CSF LE (Rho=0.823; p ≤0.001). Therefore, it is possible to infer that the instruments evaluate the same variable, carrying an almost perfect correlation and excellent construct validity.

Table 1
Correlation between the Adapted Compressed Speech Tests and Compressed Speech with Figures per ear

	N	P-value	Rho
ACS RE	22	≤0.001*	0.843
CSF RE
ACS LE	22	≤0.001*	0.823
CSF LE

• Statistical test: Spearman
• ^*
  = statistical significance; Rho = correlation coefficient
• Caption: ACS = Adapted Compressed Speech; CSF = Compressed Speech with Figures; RE = right ear; LE = left ear; N = number of subjects

For criterion analysis, it is important to highlight that there was no association between gender and groups with typical and atypical phonological development, G1 and G2, respectively (p=0.341). It is also worth noting that all children presented mild phonological disorders.

Table 2 presents the comparison of performance between groups in the CSF test. It is possible to observe that in both groups, the test did not show a significant difference between the percentages of RE and LE. This demonstrates that the instrument does not seem to differentiate populations with and without mild phonological disorders, that is, both groups have similar performance. However, it can be seen that G2 has a slightly lower average number of correct answers than G1.

Table 2
Description and comparison of performance between groups with typical (G1) and atypical (G2) phonological development

Group	N	Variable	Mean	SD	P-value
G1	22	CSF RE (%)	100.0	-	0.157
		CSF LE (%)	99.6	1.2
G2	8	CSF RE (%)	96.5	2.6	0.564
		CSF LE (%)	96.0	2.1

• Statistical test: Wilcoxon
• Caption: G1 = Group 1; G2 = Group 2; N = number of subjects; CSF = Compressed Speech with Figures; RE = right ear; LE = left ear; SD = standard deviation

In the inter-evaluator reliability analysis, it was not possible to apply statistical comparison tests, since the scoring of the protocols performed by the evaluators were 100% compatible. This demonstrates that the protocol is easy to analyze and score.

DISCUSSION

The union of language studies and CAP brought great gains for professionals and patients in these areas(¹,⁹,¹⁶). It is possible to see an increase in the number of studies that prove that the production of intelligible speech depends both on programming and motor execution capabilities, as well as on the ability to process paradigms of the acoustic process. Therefore, it is clear that there is an intimate relationship between acoustic perception and the production of speech sounds. Today it is clear that delays in the maturation stages of auditory skills can be a predictive factor for disorders in the development of speech and oral language, hence the importance of assessing them early(¹,⁹,¹⁶).

A recent study(¹⁷) serves as a basis for understanding that nowadays, the procedures adopted in a construction and validation process must be rigorous and based on scientific evidence. International literature(⁷,¹⁸) points out that it is essential that instruments used for diagnostic purposes undergo psychometric validity and reliability studies and suggest that only after these studies should they be made available for use.

In 2004, Rabelo developed the Compressed Speech Test for Brazilian Portuguese, due to the impossibility of evaluating auditory closure ability with time compression in Brazil(⁴). As mentioned above, the test was widely used in clinical practice and scientific research, however, its application to subjects with speech disorders became a major dilemma, due to the requirement for verbal responses. Therefore, the present study sought, after the content validation stage, to perform construct and criterion validation, as well as reliability analysis to release the instrument to the scientific community and enable new studies on the topic.

Construct validity is related to the degree to which an instrument is measuring the construct of interest. This validity is the most complex and difficult to determine, since it studies the degree to which the measurement scores relate to other scores of conceptually related constructs(¹⁹). In the present study, convergent validity was used, that is, the application of a correlation test between the measurements of the proposed instrument and a gold standard test (Table 1). Thus, it can be inferred that the instrument really evaluates what it proposes, due to the strong correlation between the new instrument and the existing test.

Criterion validity seeks to verify whether the instrument is truly capable of detecting alterations(¹⁹). The comparison between the results of Compressed Speech with Figures in groups with typical and atypical phonological development demonstrated that even children with mild phonological disorders obtained satisfactory results (Table 2). It is believed that the lack of a statistically significant relationship can be explained, in part, by the constitution of the group of subjects evaluated. In the sample there was no variation in the severity of the phonological disorder, with all children presenting a little altered system.

Furthermore, the test under analysis presents a great difference in relation to existing ones, the visual support material, normally understood as an easier instrument (closed set). An adaptation study that carried out all the validity and reliability stages suggested that its closed set protocol be applied when individuals had low performance in the open set application, also emphasizing this difference between the materials(²⁰). Therefore, it is understood as a limitation of the present study that it was applied to only a sample of children with low difficulty, making it necessary to use the instrument in other populations, with different degrees of phonological disorder and even other associated pathologies.

Furthermore, an instrument is considered reliable when it consistently reproduces the results applied on different occasions or by different evaluators, representing one of the main measurement properties. Therefore, the present study used inter-evaluator analysis as a basis. Some studies(²⁰,²¹) also considered it essential to analyze the vulnerability of the instrument to sources of error, which constitute threats to the validity of the test. The present study, as well as the studies mentioned above, presented unanimous results among the evaluators and, therefore, reliable.

Thus, following psychometric precepts, it was possible to complete the stages of construct validation, criterion validation and reliability analysis. The authors suggest that, in future studies, the instrument be applied to different samples with different degrees of phonological disorder, pathologies and paired groups, in order to qualify and analyze their responses.

CONCLUSION

It was possible to obtain evidence of construct, criteria and reliability for the Compressed Speech with Figures instrument. The construct analysis showed that the instrument measures the same variable as the other standard test, with an almost perfect correlation. In the criterion analysis, both groups presented similar performance, demonstrating that the instrument does not seem to differentiate populations with and without mild phonological disorders. The reliability analysis demonstrated that the protocol is easy to analyze and score.

• Study conducted at Universidade Federal de Santa Maria - UFSM - Santa Maria (RS), Brasil.
• Financial support: nothing to declare.

REFERÊNCIAS

• ¹
  Dillon H, Cameron S. Separating the causes of listening difficulties in children. Ear Hear. 2021;42(5):1097-108. http://dx.doi.org/10.1097/AUD.0000000000001069 PMid:34241982.
  » http://dx.doi.org/10.1097/AUD.0000000000001069
• ²
  Katz J, Wilde L. Desordens do processamento auditivo. In: Katz J. Tratado de audiologia clínica. 4. ed. São Paulo: Manole; 1999. p. 486-98.
• ³
  Martins JH, Alves M, Andrade S, Falé I, Teixeira A. Auditory processing disorder test battery in European Portuguese: development and Normative Data for Pediatric Population. Audiology Res. 2021;11(3):474-90. http://dx.doi.org/10.3390/audiolres11030044 PMid:34562882.
  » http://dx.doi.org/10.3390/audiolres11030044
• ⁴
  Rabelo CM. Processamento Auditivo: Teste de fala comprimida em português em adultos normo-ouvintes. [tese]. Universidade de São Paulo; 2004.
• ⁵
  Sanguebuche TR. Teste de Fala Comprimida com Figuras: construção e validação do instrumento [doutorado]. Universidade Federal de Santa Maria; 2023.
• ⁶
  Echevarria-Guanilo ME, Gonçalves N, Romaniski PJ. Propriedades psicométricas de instrumentos de medidas: bases conceituais e métodos de avaliação - parte I. Texto Contexto Enferm. 2017;26(4):e1600017. http://dx.doi.org/10.1590/1980-265x-tce-2017-0311
  » http://dx.doi.org/10.1590/1980-265x-tce-2017-0311
• ⁷
  Kirk C, Vigeland L. A psychometric review of norm-referenced tests used to assess phonological error patterns. Lang Speech Hear Serv Sch. 2014;45(4):365-77. http://dx.doi.org/10.1044/2014_LSHSS-13-0053 PMid:25091265.
  » http://dx.doi.org/10.1044/2014_LSHSS-13-0053
• ⁸
  Alexandre PD, Beber BC, Dias RF. Erros Residuais de Fala - estudo preliminar sobre características dos sistemas fonético/fonológico em falantes do Português Brasileiro. Distúrb Comun. 2020;32(4):649-57. http://dx.doi.org/10.23925/2176-2724.2020v32i4p649-657
  » http://dx.doi.org/10.23925/2176-2724.2020v32i4p649-657
• ⁹
  Sahlén B, Brännström KJ, Lyberg Åhlander V, Rudner M. Children listen: psychological and linguistic aspects of listening difficulties during development. Front Psychol. 2020;11:584034. http://dx.doi.org/10.3389/fpsyg.2020.584034 PMid:33192912.
  » http://dx.doi.org/10.3389/fpsyg.2020.584034
• ¹⁰
  Felício CM, Folha GA, Gaido AS, Dantas MMM, Azevedo-Marques PM. Protocolo de avaliação miofuncional orofacial com escores informatizado: usabilidade e validade. CoDAS. 2014;26(4):322-7. http://dx.doi.org/10.1590/2317-1782/201420140021 PMid:25211692.
  » http://dx.doi.org/10.1590/2317-1782/201420140021
• ¹¹
  Ceron MI, Gubiani MB, Oliveria CR, Keske-Soares M. Instrumento de Avaliação Fonológica (INFONO): estudo piloto. CoDAS. 2020;32(4):e20190105. http://dx.doi.org/10.1590/2317-1782/20202019105 PMid:32756856.
  » http://dx.doi.org/10.1590/2317-1782/20202019105
• ¹²
  Northern JL, Downs MP. Hearing in children. USA: Lippincott Williams & Wilkins; 2002.
• ¹³
  WHO: World Health Organization [Internet]. Prevention of blindness and deafness. Geneva: WHO; 2020 [citado em 2020 Mai 28]. Disponível em: http://www.who.int/publications-detail/basic--ear-and-hearing-care-resource
  » http://www.who.int/publications-detail/basic--ear-and-hearing-care-resource
• ¹⁴
  Jerger J. Clinical experience with impedance audiometry. Arch Otolaryngol. 1970;92(4):311-24. http://dx.doi.org/10.1001/archotol.1970.04310040005002 PMid:5455571.
  » http://dx.doi.org/10.1001/archotol.1970.04310040005002
• ¹⁵
  Folgearini JS, Goulart LLA, Silva DD, Vellozo FF, Mezzomo CL, Garcia MV. Teste de fala comprimida: adaptação e validação. Rev CEFAC. 2016;18(6):1294-301. http://dx.doi.org/10.1590/1982-021620161861816
  » http://dx.doi.org/10.1590/1982-021620161861816
• ¹⁶
  Marchetti PT, Dalcin LM, Balen AS, Mezzomo CL. Processamento auditivo temporal e os traços distintivos de crianças com transtorno fonológico. Rev CEFAC. 2022;24(3):e2022. http://dx.doi.org/10.1590/1982-0216/20222432022s
  » http://dx.doi.org/10.1590/1982-0216/20222432022s
• ¹⁷
  Luís C, Abrantes A, Oliveira C, Alves M, Martins JH. Desenvolvimento e validação de conteúdo de um Programa de Intervenção em Processamento Auditivo para crianças em idade escolar. CoDAS. 2023;35(1):e20210146. http://dx.doi.org/10.1590/2317-1782/20212021146pt PMid:36327393.
  » http://dx.doi.org/10.1590/2317-1782/20212021146pt
• ¹⁸
  Mcleod S, Verdon S. A review of 30 speech assessments in 19 languages other than English. Am J Speech Lang Pathol. 2014;23(4):708-23. http://dx.doi.org/10.1044/2014_AJSLP-13-0066
  » http://dx.doi.org/10.1044/2014_AJSLP-13-0066
• ¹⁹
  Polit DF, Yang FM. Measurement and the measurement of change. Philadelphia: Wolters Kluwer; 2016.
• ²⁰
  Araújo MEB, Lima MCO, Carvalho WLO, Brazorotto JS. Adaptation of the Brazilian Functional Auditory Performance Indicators - Short Version. CoDAS. 2021;33(1):e20190261. http://dx.doi.org/10.1590/2317-1782/20202019261 PMid:33886746.
  » http://dx.doi.org/10.1590/2317-1782/20202019261
• ²¹
  Ceron MI, Gubiani MB, Oliveira CR, Keske-Soares M. Evidence of validity and reliability of a phonological assessment tool. CoDAS. 2018;30(3):e20170180. http://dx.doi.org/10.1590/2317-1782/20182017180 PMid:29972445.
  » http://dx.doi.org/10.1590/2317-1782/20182017180

Publication Dates

• Publication in this collection
  25 Mar 2024
• Date of issue
  2024

History

• Received
  15 Mar 2023
• Accepted
  20 June 2023