Evaluation of an educational technology regarding clinical evaluation of preterm newborns

Abstracts

OBJETIVO: Evaluar un software educativo desarrollado como una herramienta para ayudar a profesores y estudiantes de enfermería neonatal en la enseñanza y el aprendizaje a través del punto de vista y evaluación por parte de los estudiantes. MÉTODO: El estudio consiste en la evaluación del contenido y las simulaciones abordadas en el software. Participaron en total 57 estudiantes de pregrado universitario afiliados a cinco universidades públicas brasileñas. RESULTADOS: La evaluación de la impresión general que tuvieron los estudiantes respecto del software fue altamente satisfactoria, ya que el 82,4% de la muestra consideró el software como una herramienta muy importante para el proceso de enseñanza. La mayoría del contenido fue considerado como bueno o muy bueno. Las sugerencias y comentarios de los participantes fueron considerados, lo cual fue crucial para la mejora y la adaptación de la nueva versión del software. CONCLUSIÓN: dentro del los métodos activos del marco pedagógico, los resultados muestran que el producto es adecuado para su uso en la enseñanza de enfermería neonatal y para la formación de técnicas de enfermería para examen físico y semiología de los recién nacidos prematuros.

Educación en Enfermería; Enfermería Neonatal; Prematuro; Validación de Programas de Computación

OBJETIVO: Avaliar um programa de computador educacional, a partir da visão dos alunos, desenvolvido como uma ferramenta para auxiliar professores e estudantes no ensino e aprendizagem sobre o tema enfermagem neonatal. MÉTODO: O estudo consiste na avaliação de conteúdo e das simulações contempladas no programa de computador. No total, participaram 57 graduandos em enfermagem vinculados a cinco universidades públicas brasileiras. RESULTADOS: A avaliação de impressão geral do software foi altamente satisfatória, sendo que 82,4% da amostra caracterizaram o programa de computador como muito importante para o ensino. A maior parte do conteúdo foi avaliada como boa ou muito boa. Foram consideradas as sugestões e comentários dos participantes como cruciais para o aprimoramento e adequação do software em sua nova versão. CONCLUSÃO: Os resultados mostram que o produto está adequado para ser utilizado no ensino de enfermagem neonatal e capacitação de enfermeiros sobre a semiotécnica e semiologia do recém-nascido pré-termo, inserido no referencial pedagógico das metodologias ativas.

Educação em Enfermagem; Enfermagem Neonatal; Prematuro; Validação de Programas de Computador

ORIGINAL ARTICLE

Evaluation of an educational technology regarding clinical evaluation of preterm newborns¹

Luciana Mara Monti Fonseca^I; Natália Del'Angelo Aredes^II; Adriana Moraes Leite^I; Claudia Benedita dos Santos^III; Regina Aparecida Garcia de Lima^IV; Carmen Gracinda Silvan Scochi^IV

^IPhD, Professor, Escola de Enfermagem de Ribeirão Preto, Universidade de São Paulo, WHO Collaborating Centre for Nursing Research Development, Ribeirão Preto, SP, Brazil

^IIDoctoral student, Escola de Enfermagem de Ribeirão Preto, Universidade de São Paulo, WHO Collaborating Centre for Nursing Research Development, Ribeirão Preto, SP, Brazil

^IIIPhD, Associate Professor, Escola de Enfermagem de Ribeirão Preto, Universidade de São Paulo, WHO Collaborating Centre for Nursing Research Development, Ribeirão Preto, SP, Brazil

^IVPhD, Full Professor, Escola de Enfermagem de Ribeirão Preto, Universidade de São Paulo, WHO Collaborating Centre for Nursing Research Development, Ribeirão Preto, SP, Brazil

^{Corresponding Author}

ABSTRACT

AIM: To evaluate, from the students' point of view, educational software developed as a tool to help teachers and students in neonatal nursing.

METHOD: The study evaluates the contents and simulations addressed in the software. A total of 57 undergraduate nursing students affiliated with five Brazilian public colleges participated.

RESULTS: The general assessment of the software was highly satisfactory: 82.4% of the sample characterized the software as quite effective as a teaching tool. Most components were assessed as good or very good. The participants' suggestions and comments are being considered in the improvement and adaptation of the new software version.

CONCLUSION: The results show that the product is adequate for use in neonatal nursing courses and nursing training on the physical examination techniques and semiology of preterm newborns, falling within the pedagogical framework of active methods.

Descriptors: Nursing Education; Neonatal Nursing; Premature; Software Validation.

Introduction

In education, the computer can be considered much more than a modern device and has activities that extend beyond creating documents or task organization. It is also an important tool to support the teaching and learning processes because it facilitates the accomplishment of different activities, enhances the understanding of complex concepts through advanced visual resources, mediates knowledge construction and serves as an assessment mechanism⁽¹⁾.

The use of educational technology is enhanced by its many advantages, including enhancing understanding of the themes being studied⁽²⁾, respecting individual rhythms, permitting repetition for the number of times necessary to understand the material⁽³⁾ and offering immediate or rapid feedback⁽⁴⁾. Although these advantages are acknowledged, educational software must nevertheless be assessed to guarantee quality.

Although many advantages are attached to educational software, information technology use in education should be a complementary tool that helps, but never replaces, teachers⁽⁵⁾.

In the attempt to guarantee more efficient learning, education has gone through constant reform, improving technology in a progressively more motivating and interactive pedagogical relation⁽⁶⁾.

Thus, in view of the countless possibilities for computer use in education, especially to remedy the existing gap left by the lack of educational materials on the clinical assessment of preterm newborns, we previously developed software stored and distributed on a CD-ROM. Its contents offer a holistic approach to clinical assessment and the techniques and strategies directed toward this goal in nursing practice in addition to using an innovative format in the teaching and learning process⁽⁷⁾.

The software contents were organized into four components: presentation, which also includes the justification and aim of the educational software and a list of abbreviations used; physical investigation techniques, containing theoretical and practical concepts of the inspection, palpation, percussion and auscultation techniques used in the clinical assessment of preterm infants; and semiology, addressing conceptual and historical aspects, the clinical assessment context (the preterm infant, the evaluator, the environment and the family) in the neonatal unit and the assessment types (birth, transition and systematic).

Systematic clinical assessment is organized according to basic human needs (oxygenation, circulation, thermoregulation, nutrition and hydration, elimination, tissue integrity, sleep and rest, sexuality, sensory perception, psychosocial and psycho-spiritual needs), which represents a novel approach to the organization of the curriculum for nursing students. The simulations involve multiple-choice questions and respective feedback on the right or wrong answer, with the goal of testing the knowledge gained by using the software. The 143 simulations are presented in a random sequence, and the student's learning is considered adequate if at least 70% of the questions are answered correctly⁽⁸⁾.

After the development phase, the software was submitted to content and face validation, involving five technological and 11 academic and clinical nursing experts. The results show that most of the items assessed garnered approval ratings higher than 70%; for example, the results demonstrated 93.7% satisfaction regarding picture, video and text quality; 81.2% positive assessment of the sound used in the program; and 81.8% approval with regard to the software's easy access. After face and content validation and after considering the technology and nursing experts' suggestions, the software was improved to guarantee better navigation and a more modern layout⁽⁷⁾.

Existing experiences in technology development and its use in health and especially in nursing education suggest the need to assess not only the constructed technological products but also their contribution to the teaching and learning process.

In this research project, the educational software was validated by the main target population: nursing students.

Aim

To evaluate the educational software "Physical Examination Techniques and Semiology of Preterm Newborns" by a group of Bachelor's and Teaching Diploma Degree students from Brazilian public colleges.

Method

The study consists of students' evaluation of an educational computer program's content and face validity, including texts, pictures, videos and simulations. It is a transversal and descriptive study and was statistically analyzed by the frequency of responses.

The proposed student evaluation verifies students' understanding of the concepts in the questionnaires and the contents of the software. This type of evaluation seeks to identify comprehension and acceptance problems with terms used from the perspective of the individuals involved in the research and to identify cognitive difficulties or other problems that must be addressed⁽⁹⁾.

This investigation is important because it measures students' satisfaction with the software and identifies areas of improvement; this software was developed to be a creative and helpful educational tool that provides new resources for students and teachers engaged in the teaching and learning process.

The participation of these students in assessing the software was appropriate in view of the need to present nursing students and teachers with new teaching and learning strategies that encourage them to create and use active and participatory educational materials.

Of 578 Brazilian undergraduate nursing programs, 29 are public Bachelor's and Teaching Diploma Degree in Nursing offered in 2009 at 18 Brazilian public higher education institutions. When selecting these Bachelor's and Teaching Diploma Degree in Nursing per state, Brazil offers one program in each of the following states: Amapá, Rio de Janeiro, Pernambuco, Sergipe, Distrito Federal, Rio Grande do Sul; two courses in the state of Paraíba; two in Paraná; four in Rio Grande do Norte; five in São Paulo and eight in Minas Gerais⁽¹⁰⁾.

The study sample derives from institutions that offer Bachelor's and Teaching Diploma Degree in Nursing. One university was drafted to represent each state, and when more than one program was offered at the same institution, only one was randomly chosen to participate.

Of 12 higher education institutions that participated in this study, five (42%) returned the completed data collection instruments in time. The remaining institutions invited to participate in the study were unable to return the completed instruments, which they justified by saying that there were general problems at their universities. The main problems described were (n=1) the health license of the responsible teacher regarding the children's health course, (n=3) federal university strikes and (n=2) the workload of the responsible teacher of the course at the time of data collection. Even with these problems, the study sample was nevertheless representative because the study represents several Brazilian regions and the universities were all public, adhering to the guidelines of the investigation.

It is important to highlight that the non-participation of the students from the non-participating universities was only because the university was not participating in the study; it says nothing about the attitudes of the students. In the cases described above, the students did not even know of the study's existence because the university did not mention it.

The 57 nursing students included in the study had previously taken the neonatal nursing care course and were interested in a more accurate assessment of their skills.

Student evaluation of educational software is fundamental because, despite previous expert validation, the target public must recognize these materials as tools that support the teaching and learning process or identify this type of educational technology as not valuable, thus avoiding the wasted expenses of money, time and human resources.

The DISABKIDS Group method was used for the evaluation⁽¹¹⁾. This group uses instruments to assess the more general aspects and acceptance of the existing items as well as the relevance of questions, student understanding and suggestions to enhance the users' understanding⁽¹²⁾.

After using the software, students completed one of the questionnaires containing questions about the students' general impression of the program, its contents and the simulations. Responses to the questions included assessments and suggestions addressing difficulties or ease of use, different formulation formats, the importance of the topic from the student's perspective and the possibility of including or modifying simulations. Objective questions were included with pre-established possible answers as well as a space for the students to make suggestions or comments. Similar methods have been used in previous studies that also had the intent of evaluating software acceptance among nursing students⁽¹³⁾.

To evaluate the contents, the 20 software topics were divided into four subgroups: subgroup 1 - presentation, justification, aims, physical examination techniques, and semiology; subgroup 2 - clinical assessment, birth assessment, transition assessment, systematic clinical assessment, and oxygenation need; subgroup 3 - circulation need, thermoregulation need, nutrition and hydration need, elimination need, and tissue integrity need; subgroup 4 - sleep and rest need, sexuality need, sensory perception need, psychosocial need, and psycho-spiritual need. Each sector was assessed for objectivity, vocabulary, text clarity, topic content presentation and instruction sequence.

For assessment, the simulations were divided into five subgroups: subgroup 1, simulations 1 to 30; subgroup 2, simulations 31 to 60; subgroup 3, simulations 61 to 90; subgroup 4, simulations 91 to 120 and subgroup 5, simulations 121 to 143.

This subdividing avoided students' having to analyze the total number of software simulations , which would have been tiresome and time-consuming and could have turned the study participants' assessment into a mechanical act. We asked if the students found the simulations relevant to learning about the topic, if they had any difficulty understanding the question format or contents and whether the response alternatives were appropriate to the simulations.

The questionnaire answers were processed by the double-entry technique, avoiding possible transcription errors, and descriptive statistics were applied using absolute frequencies and percentages.

The project was submitted to the Institutional Review Board at EERP/USP and approved under protocol n. 0974/2008.

In compliance with legislation for research involving human beings, CNS Resolution 196/96⁽¹⁴⁾, before data collection, the subjects received information about the project, their cooperation was requested, and after their acceptance, they signed an informed consent form. The consent form guaranteed secrecy and anonymity, the freedom to drop out at any time, the subjects and institutions' authorization for data collection and dissemination of the results at events and in scientific publications.

In the data collection instruments, the participants were identified by an individual codename.

Results

The sample was mostly female (96.5%) and ranged in age from 19 to 24 years (64.9%/n=37).

Of all the students from the five universities who participated in the study, 64.9% used a computer at home, 12.3% (n=7) at home and in college, and 8.8% in college only. The remaining responses referred to students who used the computer in work environments, lan houses or at the homes of relatives or friends.

Of the sample, 89.4% of the sample said they used a computer frequently, and 3.5% said they used a computer sometimes; however, none of the students said they "never" used a computer.

The students' general assessment of the software was highly satisfactory. All participants considered the software contents to be thematically important, and 82.5% of them found it extremely important. As for the questions' (simulations) importance, 98.2% found them important for learning, and 82.5% characterized them as "very important." The remaining 1.8% did not answer this question. The question, "What did you think about the software in general?" was answered by 43.9%, all of whom considered it "very good" or "good." The limited number of answers is most likely because of the participants' interpretation of this question's layout, which differed from those of the other questions. The question inquiring about their overall opinion of the software was the first question, and it was shorter than the others, which might have suggested that the question and its response alternatives were the title of the table.

Software navigation is a fundamental area for assessment. Most participants did not experience navigation problems (75.4%), but 19.3% affirmed some difficulty in this area, e.g., difficulty in identifying certain topics, returning to earlier pages to access simulator sounds and problems and interpreting some icons.

To improve the quality of the software "Physical Investigation Techniques and Semiology of Preterm Newborns" using the data collection instrument, we asked if the students would change anything in the program. In response, 45.6% answered positively and suggested less content on each page, more objective language, simulations separated by topic (similar to the content organization) and layout changes with a view to greater software dynamics.

Some students' considerations concerned specific characteristics of the virtual object, such as many links in one of the texts; the very small font of one figure, which impaired reading; and complex language in some explanatory texts. These assessments affected the validation results displayed in Table 1; however, the results still show highly satisfactory responses.

Oxygenation, sexuality, sensory perception and psycho-spiritual needs reached the highest levels satisfaction for all aspects assessed, demonstrating excellent results in terms of user acceptance and use for teaching purposes. In the area of psycho-spiritual need contents, one of the students applauded the organization of this section and found it practical and easy to understand.

The analysis of Table 1 reveals participants' high satisfaction rates with the observed aspects, with only one frequency below expected levels (70%). This area, concerning circulation and nutrition/hydration needs, thus requires reformulation because the responses indicated a lack of clarity. The other study proposal involved the students' assessment of the software simulations. The purpose of this research was to assess the relevance, interpretation and coherence of the various responses from the students' perspective. In total, 47.4% students submitted answers regarding the relevance of the simulations, 90.1% of whom found them relevant, although 8.9% manifested doubt and 1% considered them irrelevant.

Most students (89.1%) experienced no difficulties in understanding the simulations or any problems regarding the coherence between statement and response alternatives or the clarity of these alternatives (91.5%).

Among the 143 simulations, only 16 students had problems on the three simulations assessed.

In the area for suggestions and criticism, most students' comments referred to the possibility of reformulating some simulations and perhaps including figures or additional multimedia resources to clarify the simulations and render them more interesting to solve.

The researcher is reviewing the virtual object with the help of an informatics professional to adapt, correct and modify the items assessed as unsatisfactory in the semantic validation process based on the comments and suggestions the validation participants expressed. The adaptations will be incorporated into the educational software before the onset of the second study phase, which is the assessment of the nursing students' learning and knowledge retention when using the program in a virtual learning environment.

Discussion

A predominance of female participants is expected in a nursing program, as is the age range characteristic of the entry into public higher education, according to the general student profile⁽¹⁵⁾.

The high frequency of exclusive home computer use illustrates the current situation of having a computer for family or individual use and the computer's great importance in people's routines, in this case, the routines of college students. These data are similar to those of the research participant profile⁽¹⁶⁾, which assessed nursing students' computer access and knowledge and found that 59.3% of the total sample (n=123) used a computer more frequently at home. The same study observed that 55.3% (n=68) typed papers "always" and 35% (n=43) "frequently".

As for the opinions manifested in the discourse part of the data collection instrument, we reviewed the literature and found similar opinions emphasizing the visual advantages the technological resources offer, including graphs, colors, scientific papers, interaction with peers and faculty, immediate or rapid response, figures, videos and simulations. Thus, some authors also report the need for well-planned and multiprofessional (designers, programmers and specific content experts) program development work with a view to guaranteeing the users' interaction with the software, easy navigation and information access, high graphic quality, exit points that grant users flexibility to conclude the tasks and planning responses to students in case of doubts^(2,17).

The suggestions tended toward visual support. In addition, comments were made regarding the sounds; this aspect was considered important, and participants suggested the inclusion of new sounds, which is especially important given the users' satisfaction with the resource when studying auscultation. In assessing simulation by web technology⁽¹⁸⁾, the participants' answers showed the importance of sound in simulation.

These evaluation results indicate the need to improve the clarity of certain portions of the text, demonstrated by the frequency of satisfaction with the item "text clarity" for "circulation" and "nutrition/hydration" needs (see Table 1). The conciseness and clarity of digital texts are decisive elements for satisfying and significant learning, which extend beyond the ergonomic interface aspects⁽¹⁹⁾ and the guidelines⁽²⁰⁾ for web content accessibility.

Concerning the simulations, the students experienced some problems with the statements. Although this did not affect the validation in quantitative terms because few students were involved (high frequency of satisfaction with the relevance of simulations for learning physical examination techniques and semiology of preterm newborns, low levels of comprehension difficulties and incompatibility of response alternatives), special attention is due because this regards students' greater participation in their own teaching and learning processes. The students' suggestions included text reformulation but not the questions' themes or layout. Reasonable suggestions will be incorporated in the reformulation of the educational software with a view to improving nursing students' learning outcomes.

The need to analyze the usage scenario is quite important in the adaptation process of truly educational interfaces and the verification of favorable situations for teaching⁽²¹⁾ because of the need for compatibility between user expectations and the resource or intent of the digital educational material to create attractive and effective software as an auxiliary educational resource.

After informatics, audiovisual and neonatal nursing expert assessment, as described above, the software Physical Examination Techniques and Semiology of Preterm Newborns was submitted to a new evaluation process using nursing students, who constitute the target audience of the software.

The assessment process involving students was quite an important step in improving this educational software through comments, criticism and suggestions. The software consolidates its relevance to the teaching and learning process because the students, the public at whom the software is aimed, participated in its assessment and improvement. Other researchers have used the same strategy to assess educational technology, including an online educational site about the elaboration of the nursing work scale⁽²²⁾. That study involved 41 evaluators, including 24 students and 17 teachers, who judged aspects such as screen design, navigation speed, virtual environment, sound, images, simulations, hypertexts and feedback.

Another study also used student assessment in a virtual learning environment regarding clinical assessment. The author believed that students' participation can support pedagogical practices and proposed a redefinition of knowledge construction when this is considered necessary⁽²³⁾.

The software was also adapted to Internet language and is available in a virtual learning environment that offers more opportunities for autonomy and accessibility, which representadvantages offered by this type of educational tool; the software thus becomes an important resource in the education process^(13,24). The virtual learning environment enables software users to interact more actively with one another and with the teacher.

The results presented in this research indicate that there are new and innovative strategies to help students and teachers overcome problems such as lack of motivation, limited time and differences in learning rates and corroborates with other studies of a similar nature^(13,24). Finally, having users evaluate educational materials can encourage them in the construction and use of other innovative features in nursing education⁽²⁵⁾.

Conclusion

We believe that updates and improvements enhance undergraduate nursing education through software use based on the satisfactory validation and general assessment favorable to the application of this technological resource in the teaching of neonatal nursing.

We consider the product adequate for use in teaching neonatal nursing and in training nurses in physical examination techniques and the semiology of preterm newborns within the pedagogical framework of active methods.

The results showing the high levels of satisfaction among the research sample contribute to the consideration of educational practices that respect students' preferences and truly consider students as actively involved in their own teaching and learning.

Concerning topics relevant to the themes developed in this study, we suggest future studies investigating the effect of educational software used as supplementary tools for teaching and learning and their contributions to cognitive learning and attitudinal skills for nursing students.

Acknowledgements

To the professors of the nursing programs courses at the universities that participated in this study.

Received: Apr. 2^nd 2012

Accepted: Dec. 3^rd 2012

• 1. Adams A. Pedagogical underpinnings of computer-based learning. J Adv Nurs. 2004;46(1):5-12.
• 2. Wharrad HJ, Kent C, Allcock N, Wood B. A comparison of CAL with a conventional method of delivery of cell biology to undergraduate nursing students using an experimental design. Nurse Educ Today. 2001;21:579-88.
• 3. Blake H. Computer-based learning objects in healthcare: the student experience. Int J Nurs Educ Scholarship. 2010;7(1):1-15.
• 4. KaveevivitchaiC, Chuengkriankrai YL, Thanooruk R, Panijpan B, Ruenwongsa P. Enhancing Nursing Students' Skills In Vital Signs Assessment By Using Multimedia Computer-Assisted Learning With Integrated Content Of Anatomy And Physiology. Nurse Educ Today. 2008;29:65-72.
• 5. Silveira DT, Catalan VM, Neutzling AL, Martinato LHM. Objetos educacionais na consulta de enfermagem: avaliação da tecnologia por estudantes de graduação. Rev. Latino-Am. Enfermagem. [periódico na Internet]. set-out 2010 [acesso 24/10/2012];18(5):[09 telas]. Disponível em: http://www.scielo.br/pdf/rlae/v18n5/pt_23
• 6. Zem-Mascarenhas SH, Cassiani SHB. A criança e o medicamento: software educacional sobre administração de medicamentos em pediatria. Rev Bras Enferm. 2000;53(4):499-507.
• 7. Fonseca LMM, Leite AM, Mello DF, Dalri MCB, Scochi CGS. Semiotécnica e semiologia do recém-nascido pré-termo: avaliação de um software educacional. Acta Paul Enferm. 2008;21(4):543-8.
• 8. Fonseca LMM, Góes FSN, Ferecini GM, Leite AM, Mello DF, Scochi CGS. Inovação tecnológica no ensino da semiotécnica e semiologia em enfermagem neonatal: do desenvolvimento à utilização de um software educacional. Texto Contexto Enferm. 2009;18:549-58.
• 9. Polit DF, Beck CT, Hungler BP. Fundamentos de pesquisa em Enfermagem. 5th.ed. Porto Alegre: Artmed; 2004.
• 10. Ministério da Educação e Cultura (BR). Instituto Nacional de Estudos e Pesquisas Educacionais. Cadastro das Instituições de Ensino Superior. 2010. [acesso 03 fev 2010]. Disponível em: http://www.educacaosuperior.inep.gov.br/funcional/lista_cursos.asp
• 11. DISABKIDS Group. The disabkids questionnaires: quality of life children with chronic conditions. Lengerich: Pabst Science Publishers; 2006.
• 12. Deon KC, Santos DMSS, Reis RA, Fegadolli C, Bullinger M, Santos CB. Tradução e adaptação cultural para o Brasil DISABKIDS Atopic Dermatitis Module (ADM). Rev Esc Enferm USP. 2011;45(2):450-7.
• 13. Góes FSN, Fonseca LMM, Furtado MCC, Leite AM, Scochi CGS. Avaliação do objeto virtual de aprendizagem "Raciocínio diagnóstico em enfermagem aplicado ao prematuro". Rev. Latino-Am. Enfermagem. [periódico na Internet]. jul-ago 2011 [acesso 24/10/2012];19(4):[08 telas]. Disponível em: http://www.scielo.br/pdf/rlae/v19n4/pt_07
• 14. Conselho Nacional de Saúde (BR). Resolução nş 196/96 sobre pesquisa envolvendo seres humanos. Bioética. 1996;4(2):15-25.
• 15. Peres HHC, Meira KC, Leite MMJ. Ensino de didática em enfermagem mediado pelo computador: avaliação discente. Rev Esc Enferm USP. 2007;41(2):271-8.
• 16. Severo CL, Cogo ALP. Acesso e conhecimento dos acadêmicos de enfermagem acerca de recursos computacionais. Rev Gaúcha Enferm. 2006;27(4):516-23.
• 17. Bloomfield JG, While AE, Roberts JD. Using computer assisted learning for clinical skills education in nursing: integrative review. J Adv Nurs. 2008;63(3):222-35.
• 18. Barbosa SFF, Marin HF. Simulação baseada na web: uma ferramenta para o ensino de enfermagem em terapia intensiva. Rev. Latino-Am. Enfermagem. 2009;17(1):7-13.
• 19. Ergolist. Ferramentas para usabilidade: checklists de usabilidade segundo os critérios ergonômicos de Bastien e Scapin. [acesso 3 jan 2011]. 2008. Disponível em: http://www.labiutil.inf.ufsc.br/ergolist
• 20. W3C. Directivas para acessibilidade do conteúdo da Web 1.0. [acesso 3 jan 2011]. 1999. Disponível em: http://www.utad.pt/wai/wai-pageauth.html
• 21. Gomes AS, Tedesco PA, Castro Filho JA. Ambientes de Aprendizagem em Matemática e Ciências. In: Edla Maria Faust Ramos, organizadora. Informática na Escola: um olhar multidisciplinar, Fortaleza: Editora UFC; 2002.
• 22. Juliani CMCM, Kurcgant P. Tecnologia educacional: avaliação de um web site sobre escala de pessoal de enfermagem. Rev Esc Enferm USP. 2009;43(3):512-9.
• 23. Cogo ALP. Construção coletiva do conhecimento em ambiente virtual: aprendizagem da anamnese e do exame físico de enfermagem. [teses] Porto Alegre: Escola de Enfermagem da Universidade Federal do Rio Grande do Sul; 2009. 160 p.
• 24. Alvarez AG, Dal Sasso GTM. Aplicação de objeto virtual de aprendizagem, para avaliação simulada de dor aguda, em es¬tudantes de enfermagem. Rev. Latino-Am. Enfermagem. [periódico na Internet]. mar-abr 2011; [acesso 3 jan 2011]; 19(2):[09 telas]. Disponível em: http://www.scielo.br/pdf/rlae/v19n2/pt_02
• 25. Oliveira MS, Santos MCL, Almeida PC, Panobianco MS, AFC Fernandes. Evaluation of an educational handbook as a knowledge-acquisition strategy for mastectomized women. Rev. Latino-Am. Enfermagem. [periódico na Internet]. jul-ago 2012 [acesso 24 out 2012;20(4):[09 telas]. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-11692012000400006&lng=pt&nrm=iso

1

Supported by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP), process # 2008/11271-7.

Publication Dates

• Publication in this collection
  28 Mar 2013
• Date of issue
  Feb 2013

History

• Received
  02 Apr 2012
• Accepted
  03 Dec 2012