Notebook of experiences: a therapeutic resource in educational audiology

ABSTRACT

Purpose

To translate the My Experience Book toll into Portuguese, evaluate the translation content, readability, quality, and visual identity of the material, and make it available online.

Methods

Descriptive, cross-sectional, quantitative, and qualitative study. The procedures consisted of five stages: translation of the material; evaluation and response to questionnaire one regarding translation; determining the validity of the content; readability assessment; availability of material online and evaluation of content aimed at quality and visual identity (questionnaire two). Twenty-five professionals (audiologists and physicians) participated in the study.

Results

The translation of the notebook showed validity agreement rates greater than 90%. Readability rated the material easy to read. Among the 184 invited professionals, only 25 agreed to participate, demonstrating low adherence to the study. Most respondents agreed positively about the content and consistency of the translated material, videos, illustrative images, and captions. The material was also considered necessary by the majority of the evaluators. All audiologists reported using the instrument in clinical practice.

Conclusion

The translated toll is of great relevance. It gathers practical information to create a notebook of experiences and guidance on using the material as an additional resource to stimulate the auditory skills of children with hearing impairment.

Keywords
Hearing Loss; Child; Hearing Aids; Rehabilitation; Cochlear Implantation

RESUMO

Objetivo

Traduzir para o idioma português a ferramenta My Experience Book, avaliar o conteúdo da tradução, o nível de legibilidade, a qualidade e a identidade visual do material e disponibilizá-lo online.

Método

Estudo descritivo, transversal, de caráter quantitativo e qualitativo. Os procedimentos contaram com cinco etapas, sendo: tradução do material; avaliação e resposta ao questionário 1 referente à tradução; determinação da validade do conteúdo; avaliação da legibilidade; disponibilização do material online e avaliação do conteúdo voltado à qualidade e a identidade visual (questionário 2). Participaram do estudo três fonoaudiólogos, juízes e 22 profissionais (fonoaudiólogos e médicos otorrinolaringologistas de serviços de saúde auditiva e reabilitação).

Resultados

A tradução do material apresentou taxas de concordância de validade maiores que 90%. A legibilidade classificou o material com nível fácil de leitura. Dentre os 184 profissionais convidados, 22 concordaram com a participação, demonstrando baixa adesão ao estudo. A maior parte dos respondentes concordaram positivamente com os aspectos relacionados ao conteúdo e coerência do material traduzido, vídeos, imagens ilustrativas e legendas. O material também foi considerado importante pela maioria dos avaliadores. Todos os fonoaudiólogos referiram fazer uso do instrumento na prática clínica.

Conclusão

O material traduzido referente à ferramenta é de grande relevância, pois reúne informações práticas voltadas à criação de um caderno de experiências, bem como orientações quanto ao uso do instrumento como recurso adicional para a estimulação do desenvolvimento das habilidades auditivas e de linguagem de crianças com deficiência auditiva.

Descritores
Perda Auditiva; Criança; Auxiliares de Audição; Reabilitação; Implante Coclear

INTRODUCTION

Social interactions in children with typical hearing and language development are based on dialogue. In children with hearing loss, however, even with cognitive and affective maturity similar to their hearing peers, the dialogical hypotheses may be impaired due to the little contact and linguistic stimulation stimulated in the first years of life, resulting from auditory sensory deprivation and the presented speech unintelligibility. Scenarios like these result in difficulty in understanding the hypotheses reported by the child both in a therapeutic environment and in daily life^(1,2).

The family's participation in the therapeutic process enables knowledge of the children's routines, habits, preferences, facilities and difficulties⁽³⁾. Such information facilitates the understanding and attribution of meanings in the speech of children with a limited speech repertoire, being valuable to speech therapists in identifying the contexts in which the linguistic reports are inserted, as well as in the therapeutic planning with an offer of personalized intervention to the needs of the child and family with a view for expanding the dialogues and linguistic scenarios^(1-3).

Among the therapeutic strategies available for the development of auditory and oral language skills in a pleasurable, realistic and contextualized way for children, the use of specific tools, symbolic games and audiovisual resources, aimed at recording dialogues and daily social and family experiences stand out^(1,2).

In this context, the experience book emerged as an auxiliary material in the auditory rehabilitation process aimed at breaking down barriers and dialogical difficulties since it supports the child, while their language skills are not effective for maintaining dialogue⁽¹⁾. When guided and built by family members, the experience book can help in different ways in child development for the acquisition and development of oral language, auditory function, as well as in the increase of communicative and dialogical bonds^(4,5).

Authors suggest that the creation of the experience notebook follow some criteria, namely, creativity, sequencing, playful, systematic and constructive⁽⁶⁾. As a result, it is essential to incorporate content related to children's daily life through the use of photos with family and friends, as well as pictures and collages that remind the child about the moments experienced in those contexts, transforming it into a diary of records of real and remarkable facts. This interaction strategy reveals the fundamental role of the family, not only in the preparation and construction of the material, but also in facilitating children's learning as a result of adherence and family involvement in the therapeutic process⁽⁷⁾.

When inserted in the context of the clinical speech therapy routine, the experience book provides the therapist with a possibility of efficient stimulation that is, at the same time, playful and fun, with the ability to help the patient in the construction of language, strengthening the dialogue between them⁽⁸⁾.

With the increase in chronological age, the development of auditory and oral language skills, and the strengthening of the relationships established between the patient, the therapist and the family, it is expected that children will mature in their ability to recognize, point out, name and speak on the elements, scenarios and situations based on the models provided by the mediating adults⁽⁵⁾.

Despite the value of the experience book strategy in the clinical speech therapy routine and in the development of children with hearing impairment, this subject is still little discussed in the specialized literature.

At the same time, the impact of technological advances on the number of people who use the internet in search of information and guides about their own health condition or that of family members is verified. This reality is even greater among family members who need ongoing therapeutic support for children with some type of disability, including hearing loss since such conditions evoke concerns and at the same time, arouse hope in the search for innovative treatments. In this sense, the internet becomes an important resource for the management of health care for children and adolescents, an alternative source of information and reducing anxiety for families involved in therapeutic processes^(9,10).

Aiming to provide scientific materials with content adequate to the needs of families of children with hearing impairment, speech therapy has been dedicated to building websites and providing online tools for the dissemination of instructional and guidance content, achieving motivating results for further research⁽¹¹⁾. In this perspective, the responsibility to provide secure, reliable online information based on quality scientific sources and evidence becomes increasingly greater, considering the access of different audiences to the same information available^(12-14).

Therefore, the present study aimed to translate the My Experience Book tool into Portuguese, evaluate the content of the translation, the level of readability, quality and visual identity of the material and make it available online.

METHODS

Descriptive, cross-sectional, quantitative and qualitative study approved by the Research Ethics Committee of the Bauru School of Dentistry, University of São Paulo (FOB/USP), under opinion CAAE nº 59790416.0.0000.5417.

The procedures performed in the different stages of the research are described in Figure 1. The first stage consisted of translating the My Experience Book, a material consisting of texts, illustrations and videos on how and why to create an experience book developed by Nancy Caleffe-Schenck – Listen Foundation, available at the address: https://www.cochlear.com/intl/home/support/rehabilitation-resources/early-intervention/my-experience-book. The translation of the tool was authorized and carried out from English into Brazilian Portuguese by a translator-interpreter without prior knowledge of the material. The text was presented in PDF format and the videos were subtitled using the Windows Movie Maker program. The translated material was reviewed by the researchers in order to avoid technical terms, using words and expressions compatible with the target population.

Figure 1
Description of the procedures for carrying out the present research

In the second stage, three speech-language pathologists-judges with experience in the area of Educational Audiology and Hearing Rehabilitation were invited to participate in the analysis of the translated material. Participants signed the Informed Consent Form (ICF) and then answered questionnaire 1: “Content evaluation: aspects related to translation,” developed by the researchers and made available on the Google Forms platform.

Next, to determine the validity of the content of the translation in a quantitative way, the agreement rate of the inter-observer responses was calculated based on the formula: % agreement = number of judges who agreed ÷ total number of judges X 100 (step 3). According to the calculation guidelines, the items become equivalent when the agreement rates exceed a percentage of 90%⁽¹⁵⁾.

The fourth stage included the evaluation of the readability of the translated text, using the Flesch Reading Ease Index (FREI) calculated using the Microsoft Office Word tool. According to the test, scores from 100 to 75 points are compatible with a very easy reading; 75 to 50 points - easy; 50 to 25 points – difficult; and 25 to 0 points - very difficult⁽¹⁶⁾.

Stage five consisted of making the material available to the evaluators in an online environment. Next, 184 professionals who graduated from the distance Specialization Course in Hearing Qualification and Rehabilitation in Children (SCHQRC), promoted by FOB-USP in partnership with the Ministry of Health and the Samaritano Hospital of São Paulo were invited to evaluate the content regarding the quality and visual identity with 175 (95.2%) speech therapists and nine (4.8%) otorhinolaryngologists working in hearing health and rehabilitation services. The invitation was made via e-mail containing the consent form, the links to access the online material and guidelines for completing the questionnaire 2: “Content evaluation: aspects related to quality and visual identity”, developed by the researchers, and available in the Survey Monkey tool. Invitations were resent three times to each professional and a period of 30 days was made available to complete participation in the research.

The validation of the invited professionals' responses regarding the content, quality and relevance of the translated material was performed using the Content Validity Index (CVI)⁽¹⁷⁾. According to the criteria used by the formula available in the literature, the calculation of the CVI must occur from the sum of the percentage of items marked by the participants as “4” or “5”, divided by the total number of questions. To obtain a valid CVI classification, a minimum percentage of agreement of 90% is required⁽¹⁸⁾.

Data analyses were organized in the online data collection tool itself. Descriptive statistics and analysis of the frequency of responses were used.

RESULTS

The translated material was entitled “How to create a Notebook of Experiences: personalized stories in which the child is the main character” and organized in PDF format with a total of six pages (Figures 2 and 3). The graphic and textual content was organized according to the visual identity appropriate to the target audience (such as the choice of colors, illustrations and typography) in order to make the educational graphic material in question more engaging and providing greater engagement and motivation for the treatment. The videos received the titles: 1 – “The five E's and a notebook of experiences” (5:22 min); 2 – “Elements of a story in an experience notebook” (4:25 min); 3 – “Enrichment for an experience notebook” (5:57 min) and 4 – “Amartya's Pizza” (7:02 min).

Figure 2
Example cover of tool

Figure 3
Tool summary example

The results obtained from the application of questionnaire 1, as well as the agreement rate of the validity of the content analysis are shown in Table 1.

Table 1
Answers regarding the evaluation of the translation of the content and validity agreement rate (n=3)

Question	Answers	n	%	% Agreement
Did you already know the material My Experience Book?	Yes	1	33.3	Not measurable
	No	2	66.7
When comparing the content in English and in Portuguese, do you consider that the translation was: unsatisfactory, partially satisfactory or satisfactory?	Satisfatory	3	100.0	100.0
The translation of the PDF (Guidelines) presented itself as inviting for people to be interested in the audiovisual content?	Yes	3	100.0	100.0
Is the translation in an adequate language that is easy to understand for the population?	Yes	2	66.7	66.7
	No	1	33.3
Were the audiovisual resources (videos) satisfactory in terms of quality? (final content will be made available with subtitles)	Yes	3	100.0	100.0
Is the translation of the videos self-explanatory and easy to understand on how to create an “experience notebook?”	Yes	3	100.0	100.0
Is the translated material inviting to the target audience?	Yes	3	100.0	100.0
Is the translation of the material as a whole effective in creating an “experience notebook” that helps in stimulating the children's language during the therapeutic process?	Yes	3	100.0	100.0

• Caption: n = number; % = percentage
• Source: Prepared by the authors

The initial level of readability of the text obtained by the FREI statistics was 42 points, classifying the material as “difficult.”⁽¹⁶⁾ After replacing the technical terms and reducing the number of words in the longer sentences, a score of 55 was obtained on the FREI, classifying the text as “easy.”

Even after sending invitations to participate in the survey and answering questionnaire 2, of the 184 guests, a total of 162 (88.04%) professionals did not access the link available on the Survey Monkey platform, while 22 (11.96%) did . Of these, four (2.18%) did not agree to participate in the research and only 18 (9.78%) expressed acceptance, being: 16 (8.69%) speech therapists and two (1.09%) otorhinolaryngologists.

The average time to complete the questionnaire was 21 minutes. The characterization of the profile of the participants is presented in Table 2.

Table 2
Characterization of research participants (n=18)

Information	Description	n	%
Age	31 to 55	M = 39.1 years	-
Sex	Feminine	16	88.8
	Masculine	2	11,2
Region of professional activity	North	5	27.8
	Northeast	4	22.2
	South	3	16.6
	Southeast	5	27.8
	Midwest	1	5.6
Work with speech therapy for children and/or adults with hearing impairment	Did not work	5	27.8
	Worked with children	5	27.8
	Worked with children and adults	8	44.4
Places of performance	Hospitals	2	11.1
	Clinics	10	55.5
	Universities	3	16.7
	Others	3	16.7
Knowledge about the content evaluated	Yes	8	44.4
	No	2	11.2
	Partially	8	44.4

• Caption: n = number; % = percentage; M = mean
• Source: Prepared by the authors

Table 3 shows the results regarding the evaluation of the content of the experience book.

Table 3
Answers regarding the evaluation of the content of the translated material (n=18)

Questions	Totally disagree		Disagree		Indifferent		Agree		Totally agree
	n	%	n	%	n	%	n	%	n	%
Considering that the target audiences are speech therapists and family members of children with HI, do you consider that the text presented in the PDF is intelligible/understandable?	0	0.0	0	0.0	1	5.5	10	55.6	7	38.9
Is the content logically arranged, making it easier to understand?	0	0.0	1	5.5	0	0.0	12	66.7	5	27.8
Is the information offered enough for the elaboration of an experience notebook in the routine of children with hearing impairment?	0	0.0	0	0.0	2	11.1	9	50.0	7	38.9
Are the set of images, figures and videos relevant to the content?	0	0,0	0	0,0	0	0,0	10	55,6	8	44,4

• Caption: n = number; % = percentage
• Source: Prepared by the authors

Table 4 shows the assessments regarding the analysis of the visual quality of the translated material - PDF, videos, illustrative images and subtitles.

Table 4
Answers regarding the quality assessment of the translated material - PDF, videos, illustrative images and subtitles (n=18)

Questions	Terrible		Bad		Regular		Good		Excellent
	n	%	n	%	n	%	n	%	n	%
How would you rate the quality of this material (PDF and videos)?	0	0.0	0	0.0	0	0.0	7	38.9	11	61.1
How would you rate the quality of the illustrative images of the content made available in PDF?	0	0.0	0	0.0	1	5.5	9	50.0	8	44.5
How would you rate the quality of the videos and their respective subtitles?	0	0.0	0	0.0	2	11.1	11	61.2	5	27.7

• Caption: n = number; % = percentage
• Source: Prepared by the authors

The importance of the experience notebook in clinical routine is described in Table 5.

Table 5
Answers regarding the importance of the material (n=18)

Question	Not important at all		Not very important		Reasonably important		Very important		Extremely important
	n	%	n	%	n	%	n	%	n	%
How would you classify the elaborated material?	0	0.0	0	0.0	1	5.5	7	38.9	10	55.6

• Caption: n = number; % = percentage
• Source: Prepared by the authors

In the present study, the percentage of agreement in the Content Validity Index (CVI) was 94.48%.

DISCUSSION

The experience book presented in the present article aimed to offer professionals in the field of speech therapy one more possibility of therapeutic resource aimed at the development of hearing and language skills of children with hearing impairment in an accessible, free and online way. We also sought to encourage knowledge, access, interest, adherence and family involvement in the therapeutic process both in the clinical and home environments based on the bonds built between therapist, patient and family, resulting from the strategies explored by through the experience notebook.

The evaluation of the validity of materials in the health area aims to facilitate the work of professionals with regard to health promotion, the guidance and support provided to patients and families, with a view to offering more effective conditions of care, treatment and rehabilitation of the target population⁽¹⁹⁾. The translation of the material was well evaluated by the judges with validity agreement rates greater than 90%, as indicated by the test used⁽¹⁷⁾. These rates apply both to the aspects of language analysis, as well as the ease of understanding, the attractiveness of the instrument, the quality of audiovisual resources and the tool as a whole (Table 1).

The specialized literature points out some relevant factors to carry out the validation of materials in the health area, among them, the possibility of obtaining more complete, effective instruments and with greater scientific rigor, since the opinion of judges contributes to the improvement of quality of the material⁽²⁰⁾. In the present study, the additional suggestions acquired through questionnaire 1, considered relevant, were accepted and adjusted in the translated tool.

The information available in health instructional materials only becomes effective if understood by the target audience⁽²¹⁾. For this, it is necessary that the text presents a level of readability compatible with the cognitive abilities of the reader. Thus, when producing materials for such purposes, it is of paramount importance that health professionals direct special attention to the use of resources that facilitate reading and understanding the text, maximizing the reader's interest, such as the selection and use of words, technical expressions and meanings⁽²²⁾. The quality of texts available in the online environment demand even greater care considering the possibility and speed of reaching a variety of audiences with different ages, education and socioeconomic levels⁽¹³⁾. Additionally, in these cases, it is the responsibility of speech therapists to publish safe, reliable information based on quality scientific sources and evidence^(12-14). Analysis of the readability of the translated text in the experience notebook classified the material as an easy-to-read level.

The use of online questionnaires as a data collection instrument has become a current trend, increasingly present in the scientific community, standing out as a preference both among researchers and participants⁽²³⁾. Among the advantages of using this material are: global reach, flexibility, time savings, technological innovations, ease of data collection and tabulation, low cost, obtaining large samples and filling control mandatory. On the other hand, disadvantages are highlighted: the presence of spam, the selection and quality of the sample, the lack of skills of respondents, the need for technological resources, impersonality and low adherence⁽²⁴⁾. Regarding the last factor, the literature indicates that questionnaires in online format reach, on average, 25% of adherence⁽²⁵⁾.

Low adherence was seen in the present research. Despite the ease of access and the experience of the guests regarding the use of the internet, the return rate of professionals showed a limitation to the results, even after re-sending the invitations (three times in total), adherence to the answer to questionnaire 2, was only 22 (11.96%) participants. The justification for low adherence may be based on the lack of interest and participation of the guests, in addition to the time needed to evaluate the content, which included reading the material in PDF (six pages), analyzing the videos (22 .06 minutes), and the answer to the questionnaire (average of 21 minutes). Despite the low participation of invited professionals, the Content Validity Index obtained revealed a percentage of 94.48%, indicating a reliable relationship between the number of respondents and the responses obtained.

Similar results have been identified in other internet-related searches. A study⁽²⁶⁾ investigated the competences of serving nurses in educational institutions regarding the performance of essential public health functions, and pointed out that, of the 31 invited universities, only six (19.35%) returned contact with an indication of professional nursing experts. Of these, only one nurse (16.66%) responded to the proposed form. Another research proposed the application of an online questionnaire with technical-administrative employees and heads of education. In this one, the response rates obtained were 20.78% (272) for the 1,309 employees invited, and 13.85% (9) for the bosses who responded to the questionnaire⁽²⁷⁾.

Regarding the content and coherence of the translated material, videos and thematic addressed, intelligibility and layout of the text, audiovisual resources and information for the composition of the experience book, most evaluators responded that they agreed or totally agreed with the investigated aspects (Table 3). In an additional dissertation analysis, the participants judged the material as practical, accessible and easily understood by the public due to the approach of the different phases and possibilities for an effective stimulation. In general, the aforementioned aspects can be summarized in three main points: accessibility, ease of understanding and use of the material. Regarding this, the literature points out that health guidance content made available online can serve as an additional resource for stimulating the child, favoring therapeutic support networks and adding or disseminating new knowledge. Such aspects are valuable to guarantee the active participation of the family in the therapeutic process⁽²⁸⁾.

In the present research, most participants considered the quality of the translated material - PDF, videos, illustrative images and subtitles as good or excellent (Table 4). Furthermore, in a dissertation manner, they pointed out that the good quality, illustration and visibility of the images facilitated understanding and made reading more pleasurable. The videos, in turn, were described as well produced, with good quality and presentation of practical and real situations, which enriched the entire material. Knowing that the literature recognizes that the use of illustrations, videos and images can favor the understanding, internalization and learning of new content⁽²⁹⁾, it is considered that the way the material is organized can influence adherence to use by the target audience.

Regarding the importance of the material, 94.5% (n=17) considered the experience book to be very or extremely important (Table 5). In addition, all speech therapist participants reported using the instrument in clinical practice. The therapists added that this therapeutic resource is very important and useful since it provides the structuring of oral and written language stimulates creativity and provides an increase in family relationships with the possibility of use in all phases of therapy. Authors indicate that additional tools aimed at promoting the communicative development of children with hearing impairment, which enable them to adapt to the specific needs of each individual, such as the experience book, act as materials rich in stimulation, as they adjust to different contexts experienced by the child⁽³⁰⁾. It is noteworthy that the potential of using the experience book in speech therapy is not restricted to children with hearing impairment, but can be used in the therapeutic process of children with other alterations, for example, language disorders⁽²⁾.

CONCLUSION

The tool “How to create an experience notebook: personalized stories in which the child is the main character”, was evaluated as adequate in terms of translation with a high rate of validity agreement. Readability revealed an easy readability level. Most evaluators rated the quality and visual identity of the content as excellent, extremely important and suitable for online availability.

Such material is of great relevance since it gathers practical information aimed at creating an experience notebook, as well as guidance on the use of the material as an additional resource for stimulating the development of auditory and language skills in children with hearing impairment.

It is expected that from the present study, speech therapists, health and education professionals, parents and/or guardians and other interested parties will have access to this valuable therapeutic tool online and free of charge through the Hearing now and Always website from Cochlear®.

ACKNOWLEDGEMENTS

To CAPES for the encouragement and financial support for this research - Financing Code: 001

• Study conducted at Departamento de Fonoaudiologia, Faculdade de Odontologia de Bauru – FOB, Universidade de São Paulo – USP - Bauru (SP), Brasil.
• Financial support: Coordenação de Aperfeiçoamento de Nível Profissional Superior – CAPES - 001.

REFERÊNCIAS

• ¹
  Melo EM. Caderno de experiências no processo terapêutico da criança portadora de deficiência auditiva [dissertação]. São Paulo: Universidade Católica de São Paulo; 2000.
• ²
  Melo ME, Novaes BCAC. Caderno de experiências no processo terapêutico de uma criança portadora de deficiência auditiva. Pro Fono. 2001;13(2):242-8.
• ³
  Abbud GAC, Santos TCES. A família na clínica fonoaudiológica e psicopedagógica: uma valiosa parceria. Psicol Teor Prat. 2002;4(2):41-8.
• ⁴
  Schwartz S, Miller JEH. The new language of toys: teaching communication skills to children with special needs: a guide for parents and teachers. Bethesda: Woodbine House; 1996.
• ⁵
  Pollack D, Goldberg D, Caleffe-schenck N. Educational audiology for the limited-hearing infant and preschooler: an Auditory-Verbal Program. Springfield: Charles C. Thomas Publisher LTD; 1997.
• ⁶
  Rigolet SAN, Costa IMM. Estimulação precoce da linguagem escrita e o uso do caderno diário [gravação de vídeo]. São Paulo: Pró Fono; 1995.
• ⁷
  Beebe HH, Person HR, Koch ME. The Helen Beebe speech and hearing center. In: Ling D, editor. Early intervention for hearing-impaired children: oral options. Houlton: College Hill Press; 1984.
• ⁸
  Estabrooks W. Auditory-verbal therapy for parents and professionals. 2nd ed. Washington DC: Alexander Graham Bell Association for the Deaf; 1994.
• ⁹
  Hand F, McDowell DT, Glynn RW, Rowley H, Mortell AF. Patterns of internet use by parents of children attending a pediatric surgical service. Pediatr Surg Int. 2013;29(7):729-33. http://dx.doi.org/10.1007/s00383-013-3317-5 PMid:23615872.
  » http://dx.doi.org/10.1007/s00383-013-3317-5
• ¹⁰
  Lee K, Hoti K, Hughes JD, Emmerton L. Dr Google and the consumer: a qualitative study exploring the navigational needs and online health informationseeking behaviors of consumers with chronic health conditions. J Med Internet Res. 2014;16(12):e262. http://dx.doi.org/10.2196/jmir.3706 PMid:25470306.
  » http://dx.doi.org/10.2196/jmir.3706
• ¹¹
  Blasca WQ, Maximino LP, Galdino DG, Campos K, Picolini MM. Novas tecnologias educacionais no ensino da audiologia. Rev CEFAC. 2010;12(6):1017-24. http://dx.doi.org/10.1590/S1516-18462010005000021
  » http://dx.doi.org/10.1590/S1516-18462010005000021
• ¹²
  Corrêa CC, Silva RA, Blasca WQ. Elaboration and evaluation of contents about hearing health inserted in cybertutor. Int Arch Otorhinolaryngol. 2014;18(2):115-21. PMid:25992075.
• ¹³
  Chaves JN, Libardi AL, Agostinho-Pesse RS, Morettin M, Alvarenga KF. Telessaúde: avaliação de websites sobre triagem auditiva neonatal na Língua Portuguesa. CoDAS. 2015;27(6):526-33. http://dx.doi.org/10.1590/2317-1782/20152014169 PMid:26691616.
  » http://dx.doi.org/10.1590/2317-1782/20152014169
• ¹⁴
  Maximino LP, Zambonato TCF, Picolini-Pereira MM, Corrêa CC, Feniman MR, Blasca WQ. Development and evaluation of a blog about cleft lip and cleft palate and hearing. Int Arch Otorhinolaryngol. 2018;22(1):60-7. http://dx.doi.org/10.1055/s-0037-1603494 PMid:29371900.
  » http://dx.doi.org/10.1055/s-0037-1603494
• ¹⁵
  Wynd CA, Schmidt B, Schaefer MA. Two quantitative approaches for estimating content validity. West J Nurs Res. 2003;25(5):508-18. http://dx.doi.org/10.1177/0193945903252998 PMid:12955968.
  » http://dx.doi.org/10.1177/0193945903252998
• ¹⁶
  Silva CAT, Fernandes JLT. Legibilidade dos fatos relevantes no Brasil. RAC-Eletrônica. 2009;3(1):142-58.
• ¹⁷
  Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recomendations. Res Nurs Health. 2006;29(5):489-97. http://dx.doi.org/10.1002/nur.20147 PMid:16977646.
  » http://dx.doi.org/10.1002/nur.20147
• ¹⁸
  Alexandre MC, Coluci MZ. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saude Colet. 2011;16(7):3061-8. http://dx.doi.org/10.1590/S1413-81232011000800006 PMid:21808894.
  » http://dx.doi.org/10.1590/S1413-81232011000800006
• ¹⁹
  Áfio ACE, Balbino AC, Alves MDS, Carvalho LV, Santos MCL, Oliveira NR. Analysis of the concept of nursing educational technology applied to the patient. Rev Rene. 2014;15(1):158-65. http://dx.doi.org/10.15253/2175-6783.2014000100020
  » http://dx.doi.org/10.15253/2175-6783.2014000100020
• ²⁰
  Polit DF, Beck CT. The Content Validity Index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489-97. http://dx.doi.org/10.1002/nur.20147 PMid:16977646.
  » http://dx.doi.org/10.1002/nur.20147
• ²¹
  Hoffmann T, Worrall L. Designing effective written health education materials: considerations for health professionals. Disabil Rehabil. 2004;26(19):1166-73. http://dx.doi.org/10.1080/09638280410001724816 PMid:15371031.
  » http://dx.doi.org/10.1080/09638280410001724816
• ²²
  Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006;33(3):352-73. http://dx.doi.org/10.1177/1090198105277329 PMid:16699125.
  » http://dx.doi.org/10.1177/1090198105277329
• ²³
  Zuidgeest M, Hendriks M, Koopman L, Spreeuwenberg P, Rademakers J. A comparison of a postal survey and mixed-mode survey using a questionnaire on patients’ experiences with breast care. J Med Internet Res. 2011;13(3):e68. http://dx.doi.org/10.2196/jmir.1241 PMid:21946048.
  » http://dx.doi.org/10.2196/jmir.1241
• ²⁴
  Gonçalves DIF. Pesquisas de marketing pela internet: as percepções sob ótica dos entrevistados. Rev Adm Mackenzie. 2008;9(7):70-88. http://dx.doi.org/10.1590/S1678-69712008000700004
  » http://dx.doi.org/10.1590/S1678-69712008000700004
• ²⁵
  Marconi MA, Lakatos EM. Fundamentos de metodologia científica. 6. ed. São Paulo: Atlas; 2005.
• ²⁶
  Silva AM, Rodrigues CDS, Silva SMR, Witt RR. Utilização da técnica Delphi on-line para investigação de competências: relato de experiência. Rev Gaúcha Enferm. 2009;30(2):348-51. PMid:20027970.
• ²⁷
  Reorganiza UFSC. Isonomia para todos: relatório final. Grupo de Trabalho Reorganiza UFSC: isonomia para todos. Florianópolis: Serviço Público Federal, Universidade Federal de Santa Catarina; 2013.
• ²⁸
  Shoham S, Heber M. Characteristics of a virtual community for individuals who are d/deaf and hard of hearing. Am Ann Deaf. 2012;157(3):251-63. http://dx.doi.org/10.1353/aad.2012.1625 PMid:22978201.
  » http://dx.doi.org/10.1353/aad.2012.1625
• ²⁹
  Hayden P. Learner’s pocketbooks. 2. ed. Hampshire: Management Pocketbooks; 2005.
• ³⁰
  Veenema S, Gardner H. Multimedia and multiple inteligences. Am Prospect. 1996;29:69-75.

Publication Dates

• Publication in this collection
  05 Jan 2022
• Date of issue
  2022

History

• Received
  04 Jan 2021
• Accepted
  20 Mar 2021