VALIDATION FOR THE PORTUGUESE LANGUAGE OF THE SIMULATION DESIGN SCALE

VALIDACIÓN PARA LA LENGUA PORTUGUESA DE LA SIMULATION DESIGN SCALE

ABSTRACT

Considering the lack of tools to assess the results associated with the use of simulation as a teaching strategy, this study aimed to translate and validate to the Portuguese language the Simulation Design Scale (Student Version). A methodological study of instrument translation and validation was undertaken. After accomplishing all steps of the translation process, the validation process took place in the context of an event on critical patient care. In total, 103 registered nurses participated in the study. The validity and reliability of the scale, the pattern of correlation between variables, the sampling adequacy test and the sphericity test showed satisfactory results. As there is no connection among the groupings established in the exploratory factor analysis, it was decided to follow the division established in the original version. The scale was named Escala do Design da Simulação [in Portuguese]. It is concluded that the scale has good psychometric properties and an appropriate potential, although further research is needed for their consolidation.

Simulation; Teaching; Education, Nursing; Validation Studies

RESUMEN

Teniendo en cuenta la carencia de herramientas que evalúen los resultados asociados con el uso de la simulación como estrategia de enseñanza, este estudio metodológico de traducción y validación de instrumento tuvo como objetivo traducir y validar para la lengua portuguesa la Simulation Design Scale (Student Version). Después de completados todos los pasos del proceso de traducción, el proceso de validación se llevó a cabo en el contexto de evento acerca del cuidado del paciente crítico. Participaron del estudio 103 enfermeros licenciados. La validez y fidelidad de la escala, el patrón de correlación entre las variables, el test de adecuación de las muestras y el test de esfericidad presentaron resultados satisfactorios. Por no haber conexión entre los agrupamientos establecidos en el análisis factorial exploratorio, se optó por seguir la división establecida por la versión original. La escala fue denominada Escala do Design da Simulação [en Portugués]. Se concluye que la escala posee buenas propiedades psicométricas y potencial apropiado, sin embargo, futuras investigaciones son necesarias para su consolidación.

Simulación; Enseñanza; Educación en Enfermería; Estudios de Validación

RESUMO

Diante da escassez de instrumentos que avaliem os resultados associados à utilização da simulação como estratégia de ensino, foi objetivo deste estudo traduzir e validar para a língua portuguesa a Simulation Design Scale (Student Version). Estudo metodológico de tradução e validação de instrumento. Depois de cumpridas todas as etapas do processo de tradução, o processo de validação realizou-se no contexto de evento sobre atendimento ao paciente crítico. Participaram deste estudo 103 enfermeiros. A validade e fidelidade da escala, o padrão de correlação entre as variáveis, o teste de adequação amostral e o teste de esfericidade apresentaram bons resultados. Por não haver conexão entre os agrupamentos estabelecidos na análise fatorial exploratória, optou-se por seguir a divisão estabelecida pela versão original. A escala foi denominada: Escala do Design da Simulação. Conclui-se que a escala possuí boas propriedades psicométricas e um apropriado potencial; no entanto, futuras pesquisas são necessárias para sua consolidação.

Simulação; Ensino; Educação em Enfermagem; Estudos de Validação

INTRODUCTION

As a result of the advent of technology, nursing teaching has undergone several transformations to attend to the needs of an increasingly modern and globalized society. In view of this scenario, educational institutions have restructured their education practices, aiming to promote the education of more competent professionals who are committed to the health of the population, endowed with critical thinking and the skills needed to perform the nursing profession.^1-2

In view of the need for changes in theoretical and practical teaching, clinical simulation is a highly valued tool among the pedagogical strategies.³To comply with its teaching proposal, competent and well trained professionals are required for a good performance, as technological and modern resources alone do not guarantee the efficacy of the strategy.⁴ The use of clinical simulation in the teaching context demands well-planned and structured criteria, with an appropriate method, trained professionals and efficient resources to achieve the objectives set. The scenarios elaborated for the simulation practice should be similar to the reality and rest on well-defined scopes regarding the aspects to be developed during the simulation activity. A scenario can be considered well-elaborated and realistic when it allows the subjects physical assessment, training of technical skills and critical thinking about the nurses' role in the simulated situation.^5-7

Nowadays, there is great concern with the construction of the scenarios for simulated practice; in many institutions, there is no pedagogical script for its construction, threatening the efficacy of the activity. It is recommended⁸ that the clinical simulation strategy follows a design, containing: a) Objectives: which are related to the characteristics of the scenario to be simulated and to the description of the intentions to be achieved, which should be informed to the participants in advance; b) Fidelity: in line with the ability of the scenario to be similar to the reality. All material and equipment needed to accomplish the scenario should be available to implement the proposed simulated situation and be identified by the participants; c) Problem solving: the complexity addressed in the scenario and the content addressed in the simulated environment should be compatible with the theoretical content addressed in the classroom, with a view to strengthening the knowledge; d) Student support: the clues provided during the scenario for the participant to better understand the simulated situation. These can be provided verbally by the teacher and/or facilitator, visualized through the monitors, or even expressed by the simulator; e) Debriefing: session held after terminating the scenario, in which participants and teacher remember the positive facts and the areas for possible improvements that occurred during the activity.

In view of these parameters, programming a simulated activity demands a harsh preparation from the teacher in terms of the design of the scenario that is to be simulated. Despite good structuring by the teacher, errors and flaws can happen in the course of the activity and, if the teacher does not address and clarify them, they can compromise the success of the strategy, besides creating traumas for the participants. In that sense, many institutions have decided to validate their scenarios and have documented these scripts to standardize them.

The National League for Nursing (NLN),⁹ an American league dedicated to excellence in Nursing teaching, developed The Simulation Design Scale (student version), with a view to assessing the structuring of their scenarios. It is a 20-item tool, divided in two subscales: the first about the simulation design and second about the importance of the item for the participant. The subscales are also divided in five factors that assess: 1) The objectives and information; 2) The support; 3) The solution of problems; 4) The feedback and reflection; 5) The realism. A five-point Likert response scale is used, including the option non-applicable when the declaration does not refer to the simulated activity done.

The validation study of this tool involved 395 nursing students, 350 women and 45 men. The reliability verified using Cronbach's alpha corresponded to 0.92 for the scale of specific design characteristics and 0.96 for the importance of the resources.¹⁰

In Portuguese, there are no tools so far that assess the structuring characteristics of the scenarios to implement the clinical simulation. Hence, the objective in this study was to translate and validate for the Portuguese language a tool capable of measuring the characteristics of the simulation design and then make it available to contribute to the strengthening of this teaching tool.

METHOD

A methodological instrument translation and validation study was undertaken, which aims for the investigation of data collection and organization methods. After receiving permission from the authors of the original scale, institutional review board approval was sought and obtained under opinion 294.206/2013. To guarantee the quality of the translation and validation of the Simulation Design Scale, the study was developed in two phases.

The first phase consisted of the translation of the tool, following the method proposed by Ferrer et al.¹¹ After two official teachers had translated the tool to Portuguese, a consensus was obtained on the first version in Portuguese. The first version was submitted to an expert committee (nurses experts in fundamental nursing; experts on simulation as a teaching strategy) who, after clarifications on the research objective, manifested their acceptance by signing the Informed Consent Term. The experts were advised to classify the instrument items as valid or not, after which the Content Validity Index (CVI) was calculated;¹² items with a CVI equal to 100% had their translation maintained in the definitive tool. The language of items with a CVI below 80% was modified.

Next, the tool was submitted to back-translation by two teachers, being one official translator and another native English speaker, for the sake of comparison with the original version. After verifying that the meaning of the instrument had not been changed, the semantic validation of the tool was undertaken, along with a pretest that involved ten undergraduate students who had already experienced simulation as a teaching strategy and were invited to manifest their doubts and considerations on the understanding of the tool. It should be highlighted that none of the undergraduates who took part in this phase participated in the validation process of the tool.

The second phase of the study was the validation of the tool. For this phase, the event entitled "III Workshop Brazil-Portugal: Care Delivery to Critical Patients" was organized, promoted by a Brazilian teaching institution in partnership with a Portuguese teaching institution. The people invited to participate in this workshop were nurses, professionally active or not, with or without a graduate degree, who had previously experienced clinical simulation in their education or not. This event was totally free, disseminated in print and electronically, and 180 places were offered.

The people enrolled could choose from three alternative days to take part in the event, as each day of the workshop consisted of 60 participants. All vacancies were filled in advance and preliminary reading and study material was forwarded by e-mail. Among the 180 participants enrolled, 103 attended the event. The workshop started in the morning with a discussion of the theoretical aspects of critical patient care and the use of clinical simulation. Teachers from Brazil and Portugal with expertise in the area offered the content.

The participants were invited to participate in the research and manifested their acceptance by signing the Informed Consent Form. To characterize the subjects, the authors developed a tool with the following variables: age, sex, year of graduation, professional activity (years), educational data, employment bond and experience with simulated teaching. After explain the theoretical content, still in the morning, the participants were divided in three groups and took part in three workshops for skills training. Throughout the event, low, medium and high-fidelity teaching strategies were used, besides simulated patients.

In the afternoon, each group passed through three different simulated scenarios, all of which involved care delivery to critical patients in predetermined situations. This care involved the preliminary reading material offered by e-mail, the presentation and the skills training workshops. After complying with all phases of the event, the participants were jointed in an auditorium to complete the research tool.

The data were typed in a Microsoft Excel^(r) worksheet and analyzed using the software Statistical Package for Social Sciences (version 22 for Windows). For all tests, significance was set at α=0.05.

RESULTS

In the first phase, among the 20 instrument items, only item 14 obtained a CVI inferior to 80%, having its formulation modified: the experts suggested that "objectives for my patient" did not sound well. Hence, when reassessing the item, the word "care" was added for the sake of a better understanding (see item 14 in Table 1). After this adaptation, the entire process was executed regularly. The pilot test involving students also proceeded regularly, without difficulties to understand and answer the tool.

The final version of the scale is described in Table 1.

Table 1
- Final version of the Escala de Design da Simulação, Ribeirão Preto, São Paulo, 2014

Item
Factor 1) Objectives and information
1. No início da simulação foi fornecida informação suficiente para proporcionar orientação e incentivo.
2. Eu entendi claramente a finalidade e os objetivos da simulação.
3. A simulação forneceu informação suficiente, de forma clara, para eu resolver a situação-problema.
4. Foi-me fornecida informação suficiente durante a simulação.
5. As pistas foram adequadas e direcionadas para promover a minha compreensão.
Factor 2) Support
6. O apoio foi oferecido em tempo oportuno.
7. A minha necessidade de ajuda foi reconhecida.
8. Eu senti-me apoiado pelo professor durante a simulação.
9. Eu fui apoiado no processo de aprendizagem.
Factor 3) Solution of problems
10. A resolução de problemas de forma autônoma foi facilitada.
11. Fui incentivado a explorar todas as possibilidades da simulação.
12. A simulação foi projetada para o meu nível específico de conhecimento e habilidades.
13. A simulação permitiu-me a oportunidade de priorizar as avaliações e os cuidados de enfermagem.
14. A simulação proporcionou-me uma oportunidade de estabelecer objetivos para a assistência do meu paciente.
Factor 4) Feedback / Reflection
15. O feedback fornecido foi construtivo.
16. O feedback foi fornecido em tempo oportuno.
17. A simulação permitiu-me analisar meu próprio comportamento e ações.
18. Após a simulação houve oportunidade para obter orientação / feedback do professor, a fim de construir conhecimento para outro nível.
Factor 5) Realism
19. O cenário se assemelhava a uma situação da vida real.
20. Fatores, situações e variáveis da vida real foram incorporados ao cenário de simulação.

In the second phase, among the 103 participants in the workshop, 100% agreed to participate in the research and were included in the study sample. Most participants were female, 90 (87.4%), with 32.1 years as the mean age. As regards education, the mean undergraduate course conclusion year was 2005, and 64 (62.1%) held or were taking a Lato Sensu specialization degree; 47 (45.7%) held or were taking a Master's degree, 20 (19.4%) held or were taking a Ph.D. and 20 (19.4%) did not possess any kind of specialization.

As regards the professional activity, 77 (74.8%) had an employment contract; 48 (46.6%) of them were active in care, 23 (22.3%) in teaching and six (5.8%) were health service managers. Concerning the experience with simulated teaching, 52 (50.5%) reported being unfamiliar with clinical simulation as a teaching tool and 51 (49.5%) said they were familiar with this strategy.

The validity and fidelity of the scale, verified through the correlation matrix, showed 164 (41%) correlations higher than 0.30. The sampling adequacy test, verified using Kaiser-Meyer-Olkin, corresponded to 0.85 with Bartlett's sphericity test <0.001 and the coefficients in its anti-image matrix continued between 0.78 and 0.92.

Concerning the extraction of factors, the total explained variance showed four eigenvalues superior to 1.00, with a cumulative percentage that explained more than 69% of the total variance, suggesting that the scale could be divided in four factors.

In the rotation of the factors, the communality test evidenced that all items score over 0.6, strengthening the possible extraction in factors. To verify the factors, the original authors' approach was followed, using exploratory factorial analysis with varimax rotation. The result of this analysis was very different from the findings of the original scale; without coherence between the groups, the researchers chose to follow the division established in the original version.

The internal consistency, verified using Cronbach's alpha, has been described in Table 2, as follows.

Table 2
- Internal consistency of the Escala de Design da Simulação, n = 103, Ribeirão Preto, São Paulo, 2014

	Simulation design	Importance of Item
Factor 1	0.85	0.83
Factor 2	0.83	0.87
Factor 3	0.83	0.80
Factor 4	0.85	0.89
Factor 5	0.88	0.89
General Scale	0.93	0.94

Tables 3 and 4 present the descriptive statistical results for the Escala do Design da Simulação (Student Version) and the Importância do Item.

Table 3
- Descriptive statistics of factors of the Escala do Design da Simulação, n = 103, Ribeirão Preto, São Paulo, 2014

	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	General
Mean	4.33	4.45	4.50	4.80	4.71	4.53
Standard Deviation	0.61	0.58	0.56	0.35	0.53	0.44
Minimum	2.00	2.00	2.00	3.00	2.00	3.00
Maximum	5.00	5.0	5.00	5.00	5.00	5.00
Percentiles	25	4.00	4.00	4.00	4.75	4.50	4.20
	50	4.40	4.50	4.80	5.00	5.00	4.70
	75	4.80	5.00	5.00	5.00	5.00	4.85

Table 4
- Descriptive statistics of factors of the Escala de Importância do Item (n = 103), Ribeirão Preto, São Paulo, 2014

	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	General
Mean	4.80	4.84	4.82	4.89	4.89	4.84
Standard Deviation	0.32	0.31	0.32	0.26	0.31	0.26
Minimum	3.80	4.00	3.80	4.00	3.50	4.00
Maximum	5.00	5.00	5.00	5.00	5.00	5.00
Percentiles	25	4.80	5.00	4.80	5.00	5.00	4.80
	50	5.00	5.00	5.00	5.00	5.00	5.00
	75	5.00	5.00	5.00	5.00	5.00	5.00

Pearson's correlation coefficient¹³demonstrated, for the Escala de Design da Simulação, a strong (superior to 0.77) correlation of factors 1, 2, 3 and 4, and a moderate correlation (0.68) for factor 5. In addition, a weak correlation (0.24) was observed between the Escala do Design da Simulação and the Escala de Importância do Item, with a significance level of 0.01.

DISCUSSION

The simulation of clinical practice in health enhances the range of the professionals' self-confidence and satisfaction, which supports the effectiveness of health care.⁸In nursing teaching, its practice is expanding; hence, the aim in this study was the validation for the Portuguese language of the Simulation Design Scale (Student Version), which was called Escala do Design da Simulação.

The development of the psychometric tests indicated a high correlation among the variables and an appropriate sampling adequacy to develop the study. The total explained variance signaled that the scale could be divided in four factors, diverging from the original scale. When the exploratory factorial analysis with varimax rotation was applied, the items grouped in a very unexpected manner; as there was no logical explanation for this grouping, the choice was made to follow the findings of the original version. One justification for this unexpected grouping can be the characteristics of the study sample. The validation study of the original version was developed with undergraduate nursing students,¹⁰ and this validation with nurses. In view of the heterogeneity of these samples, the factors on this scale should be better assessed in other studies, involving students and professionals.

In this study, as the scale can be applied to any individual who uses clinical simulation for education purposes, the choice was made to use nurses as a study sample, considering that these professionals working in clinical practice also need constant professional improvement, mainly using innovative technologies directed at significant learning, mainly valuing each subject's daily reality.

Nevertheless, another limitation that may have influenced this study is the sample size. It is recommended that, for the factorial analysis, the sample should contain at least five participants per variable and a total of at least 200 subjects;¹⁴ the use of 10 subjects per item is also recommended,¹⁵ with a total of at least 100 subjects. In addition, the desired sample size depends on the size of the factor loadings, around 0.80.¹⁶ Samples of 50 individuals as very low, 100 as low, 200 as reasonable, 300 as good, 500 as very good and 1,000 or more as excellent.¹⁷ The use of 10 subjects for each instrument item is added as necessary,¹⁸ including the exemption that any factorial analysis with less than 200 items can hardly be considered appropriate. Hence, it is perceived that there is no consensus on the sample size. Based on this reality and these facts, the tests were developed with 103 individuals.

In total, 180 places were available to register for the workshop; all places were filled, but only 103 people attended the event. Factors related to the professionals' mobility, to the availability to be absent from their professional activities, among others, contributed to the incomplete attendance of the registered professionals. Nevertheless, hospital work has increasingly demanded new competences from health professionals, who are confronted with technological changes as well as with the requirements of their clients.¹⁹

As regards the internal consistency, the results found are similar to the findings in the original version,¹⁰ for the Escala do Design da Simulação as well as for the Escala de Importância do Item, reaffirming the coherence of the scales. The internal consistency of the five factors was also good in both scales.

As regards the descriptive statistics, the participants presented better means in factor 4, followed by factors 5, 3, 2 and 1. They considered factor 4 as the most important, followed by factors 5, 2, 3 and 1. Nevertheless, this kind of results should be explored in future research. Pearson's correlation test demonstrated a moderate positive correlation between most factors of the Escala de Design da Simulação, indicating convergence between the factors and the general scale, with a view to understanding the aspects of the simulation design. Nevertheless, although both scales are part of the same tool, they are weakly correlated, which is completely understandable when considering the characteristics and interpretation of each.

CONCLUSION

Teaching through simulation has consolidated as a strategy that offers different benefits, as it makes learning significant through the relation between theory and practice. Nevertheless, tools that permit assessing the strategy remain scarce. Understanding specific characteristics of how the individual assimilates the clinical simulation scenario is fundamental for the strengthening of this teaching strategy; besides contributing to the development of specific items in the strategy.

In this study, the Portuguese translation of the Simulation Design Scale (Student Version) was developed, which is called Escala do Design da Simulação in Portuguese, which can be applied in any individual who intends to enhance his/her knowledge by using this strategy.

The psychometric results found in this sample are appropriate, except in the factorial analysis, which demands future investigation. The sample size was considered as a limitation in this study, but further research will provide sustainability to consolidate the validity of the scale, as well as to support its strengthening as a pedagogical strategy.

REFERENCES

• ¹ Teixeira CRS, Kusumota L, Braga FTMM, Gaioso VP, Santos CB, Silva VLS, et al. O uso de simulador no ensino de avaliação clínica em enfermagem. Texto Contexto Enferm [online]. 2011 [acesso 2014 Nov 19]; 20(spe):187-93. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072011000500024&lng=en
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072011000500024&lng=en
• ² Silva KL, Sena RRD, Grillo MJC, Horta NDC. Formação do enfermeiro: desafios para a promoção da saúde. Esc Anna Nery Rev Enferm. 2010; 14(2):368-76.
• ³ Marmol MT, Braga FTMM, Garbin LM, Moreli L, Santos CB, Carvalho EC. Curativo de cateter central em simulador: efeito da presença do tutor ou da aprendizagem autoinstrucional. Rev Latino-Am Enfermagem [online]. 2012 [acesso 2013 Dez 17]; 20(6):1134-41. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-11692012000600016&lng=en
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-11692012000600016&lng=en
• ⁴ Howard V, Ross C, Mitchell A, Nelson G. Human patient simulators and interactive case studies: A comparative analysis of learning outcomes and student perceptions. Comput Inform Nurs. 2010 Jan-Feb; 28(1):42-8.
• ⁵ Bricker DJ, Pardee CJ. Nurse experts jump-start clinical simulation in rehabilitation nursing: supporting new graduate transition to competence. Nurs Educ Perspect. 2011 Jan-Feb; 32(1):34-6.
• ⁶ Liaw SY, Chen FG, Klainin P, O'Brien A, Samarasekera DD. Developing clinical competency in crisis event management: an integrated simulation problem-based learning activity. Adv Health Sci Educ Theory Pract [online]. 2010 Aug [acesso 2013 Dez 17]; 15(3):403-13. Disponível em: http://link.springer.com/article/10.1007%2Fs10459-009-9208-9.
  » http://link.springer.com/article/10.1007%2Fs10459-009-9208-9
• ⁷ Martins JCA, Mazzo A, Negrão RBC, Coutinho VRD, Godoy S, Mendes IAC, et al. A Simulada Experiência Clínica no Ensino de Enfermagem: Retrospectiva Histórica. Acta Paul Enferm [online]. 2012 [acesso 17 Nov 2014]; 25(4):619-25. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0103-21002012000400022&lng=en
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0103-21002012000400022&lng=en
• ⁸ Jeffries P. Simulation in nursing education: From conceptualization to evaluation. New York (US): National League for Nursing; 2007.
• ⁹ National League for Nursing (NLN). Simulation Innovation Resource Center Glossary [online]. [acesso 2013 Nov 09]. Disponível em: http://www.sirc.nln.org/mod/glossary/view.php
  » http://www.sirc.nln.org/mod/glossary/view.php
• ¹⁰ Jeffries PR, Rizzolo MA. Designing and implementing models for the innovative use of simulation to teach nursing care of ill adults and children: A national, multi-site, multi-method study. In: National League for Nursing/Leardal project summary report. New York (EUA): National League for Nursing; 2006.
• ¹¹ Ferrer M, Alonso J, Prieto L, Plaza V, Monso E, Marrades R, et al. Validity and reliability of the St George´s Respiratory Questionnaire alter adaptation to a different language and culture: the Spanish example. Eur Respir J. 1996; 9(6):1160-6.
• ¹² Polit DF, Beck CT. The Content validity index: are you sure you know what's being reported? critique and recommendations. Res Nurs Health. 2006; 29(5):489-97.
• ¹³ Zou HK, Tuncali K, Silverman SG. Correlation and Simple Linear Regression. Radiology. 2003; 227(1):617-28.
• ¹⁴ Gorsuch RL. Factor analysis. 2ª ed. Hillsdale (EUA): Lawrence Erlbaum Associates; 1983.
• ¹⁵ Crocker L, Algina J. Introduction to classical and modern test theory. New York (EUA): Holt, Rinehartand Winston; 1986.
• ¹⁶ Guadagnoli E, Velicer WF. Relation of sample size to the stability of component patterns. Psychological Bulletin. 1988; 103(1):265-75.
• ¹⁷ Comrey AL, Lee HB. A first course in factor analysis. Hillsdale (EUA): Erlbaum; 1992.
• ¹⁸ Pasquali L. Instrumentos psicológicos: Manual prático de elaboração. Brasília (Brasil): LabPAM; 1999.
• ¹⁹ Camelo Silvia Helena Henriques. Competência profissional do enfermeiro para atuar em Unidades de Terapia Intensiva: uma revisão integrativa. Rev Latino-Am Enfermagem. 2012; 20(1):192-200.

Publication Dates

• Publication in this collection
  Dec 2015

History

• Received
  16 Feb 2015
• Accepted
  17 Aug 2015