Cultural adaptation and evaluation of the measurement properties of the Facilitator Competency Rubric for clinical simulation facilitators

Campos, Daniel Gonçalves; Silva, Juliany Lino Gomes; Silva, Ruana Luiz Ferreira da; Almeida, Angélica Olivetto; Lira, Ana Luísa Brandão de Carvalho; Oliveira-Kumakura, Ana Railka de Souza

doi:10.1590/1518-8345.7214.4257

Daniel Gonçalves Campos Corresponding author: Daniel Gonçalves Campos E-mail: dangcampos@gmail.com https://orcid.org/0000-0003-0729-6441

Study concept and design

Obtaining data

Data analysis and interpretation

Statistical analysis

Drafting the manuscript

Critical review of the manuscript as to its relevant intellectual content

Approved the final version of the text

Universidade Estadual de Campinas, Faculdade de Enfermagem, Campinas, SP, Brazil.

https://orcid.org/0000-0003-0729-6441

Juliany Lino Gomes Silva

Obtaining data

Data analysis and interpretation

Drafting the manuscript

Critical review of the manuscript as to its relevant intellectual content

Approved the final version of the text

Universidade Estadual de Campinas, Faculdade de Enfermagem, Campinas, SP, Brazil.

https://orcid.org/0000-0002-2930-103X

Ruana Luiz Ferreira da Silva

Obtaining data

Data analysis and interpretation

Drafting the manuscript

Critical review of the manuscript as to its relevant intellectual content

Approved the final version of the text

Universidade Estadual de Campinas, Faculdade de Enfermagem, Campinas, SP, Brazil.

https://orcid.org/0000-0002-3144-5788

Angélica Olivetto Almeida

Obtaining data

Data analysis and interpretation

Drafting the manuscript

Critical review of the manuscript as to its relevant intellectual content

Approved the final version of the text

Universidade Estadual de Campinas, Faculdade de Enfermagem, Campinas, SP, Brazil.

https://orcid.org/0000-0001-5158-493X

Ana Luísa Brandão de Carvalho Lira

Obtaining data

Data analysis and interpretation

Drafting the manuscript

Critical review of the manuscript as to its relevant intellectual content

Approved the final version of the text

Universidade Federal do Ceará, Departamento de Enfermagem, Fortaleza, CE, Brazil.

https://orcid.org/0000-0002-7255-960X

Ana Railka de Souza Oliveira-Kumakura

Study concept and design

Data analysis and interpretation

Statistical analysis

Drafting the manuscript

Critical review of the manuscript as to its relevant intellectual content

Approved the final version of the text

Université Paris Cité, Département Universitaire en Sciences Infirmières, Paris, France.

https://orcid.org/0000-0002-7075-7987

Corresponding author:
Daniel Gonçalves Campos E-mail: dangcampos@gmail.com https://orcid.org/0000-0003-0729-6441

Conflict of interest:

the authors have declared that there is no conflict of interest.

Associate Editor:

Rosana Aparecida Spadoti Dantas

	Equivalences								Clarity		Relevance
Item - Domain	Semantic		Idiomatic		Conceptual		Cultural
	CVI ^* * CVI = Content Validity Index	*Kappa*	CVI ^* * CVI = Content Validity Index	*Kappa*	CVI ^* * CVI = Content Validity Index	*Kappa*	CVI ^* * CVI = Content Validity Index	*Kappa*	CVI ^* * CVI = Content Validity Index	*Kappa*	CVI ^* * CVI = Content Validity Index	*Kappa*
1.2 - Domain 1	0.857	0.857	0.893	0.893	0.893	0.893	0.929	0.928	0.893	0.893	1.00	1.00
1.4 - Domain 1	0.857	0.857	0.893	0.893	0.929	0.928	0.929	0.929	0.964	0.964	0.857	0.857
1.5 - Domain 1	0.893	0.893	0.929	0.929	0.964	0.964	0.964	0.964	0.929	0.929	0.964	0.964
1.6 - Domain 1	0.857	0.857	0.929	0.929	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
1.1 - Domain 2	0.893	0.893	0.964	0.964	0.964	0.964	0.964	0.964	0.964	0.964	0.964	0.964
1.2 - Domain 3	0.893	0.893	0.929	0.929	0.964	0.964	0.964	0.964	0.964	0.964	0.964	0.964
1.6 - Domain 4	0.893	0.893	0.929	0.929	0.964	0.964	0.964	0.964	0.929	0.929	1.00	1.00

Item	Pre-final version	Version resulting from the 2nd evaluation of the committee of experts (Final version)
Introductory text	No content	Dear, if you are carrying out your self-assessment or evaluating a professional using the Brazilian version of the Facilitator Competency Rubric (FCR), we inform you that the competencies for a facilitator are evaluated through five domains (preparation, prebriefing, facilitation, debriefing and evaluation). For each domain, there is a list of items. We recommend that you read each item carefully and choose the rating that best represents you or the professional you are evaluating: (1) Beginner; (2) Advanced Beginner; (3) Competent; (4) Proficient; (5) Expert At the end, you will find the expected score for each domain.
Score line in each domain	Template for the total of the three columns — “Preparation” Section 0-14 = “Beginner” to “Advanced Beginner” (needs supervision from a “Proficient” to “Expert” facilitator). 15-27 = “Competent”. 28-35 = “Proficient” to “Expert” (able to supervise a “Beginner” to “Advanced Beginner” facilitator).	Template for the total of the three columns — “Preparation” Section 0-14 = “Beginner” to “Advanced Beginner” (needs supervision from a “Proficient” to “Expert” facilitator). 15-27 = “Competent”. 28-35 = “Proficient” to “Expert” (able to supervise a “Beginner” to “Advanced Beginner” facilitator).

Scale domains	Classification	Pre-final version testing (n=33)	Reliability Test (n=52)	Cronbach’s Alpha (n=52)	ICC ^* * ICC = Intraclass correlation coefficient; (n=15)
Preparation		Average ^† † Average score per domain; =24.54 (SD ^‡ ‡ SD = Standard deviation per domain =7.30)	Average ^† † Average score per domain; =25.11 (SD ^‡ ‡ SD = Standard deviation per domain =6.25)	0.911	0.95
	Beginner to Advanced Beginner	12.1% (4)	3.8% (2)
	Competent	51.5% (17)	57.7% (30)
	Proficient to Expert	36.4% (12)	38.5% (20)
Prebriefing		Average ^† † Average score per domain; =14.69 (SD ^‡ ‡ SD = Standard deviation per domain =3.99)	Average ^† † Average score per domain; =13.82 (SD ^‡ ‡ SD = Standard deviation per domain =4.26)	0.876	0.96
	Beginner to Advanced Beginner	9.1% (3)	9.6% (5)
	Competent	42.4% (14)	55.8% (29)
	Proficient to Expert	48.5% (16)	34.6% (18)
Facilitation		Average ^† † Average score per domain; =22.42 (SD ^‡ ‡ SD = Standard deviation per domain =5,89)	Average ^† † Average score per domain; =22.82 (SD ^‡ ‡ SD = Standard deviation per domain =4.93)	0.906	0.95
	Beginner to Advanced Beginner	12.1% (4)	1.9% (1)
	Competent	27.3% (9)	53.8% (28)
	Proficient to Expert	60.6% (20)	44.3% (23)
Debriefing		Average ^† † Average score per domain; =30.51 (SD ^‡ ‡ SD = Standard deviation per domain =7.45)	Average ^† † Average score per domain; =29.56 (SD ^‡ ‡ SD = Standard deviation per domain =7.08)	0.947	0.97
	Novice to Advanced Beginner	6.1% (2)	5.8% (3)
	Competent	39.4% (13)	55.7% (29)
	Proficient to Expert	54.5% (18)	38.5% (20)
Evaluation		Average ^† † Average score per domain; =14.06 (SD ^‡ ‡ SD = Standard deviation per domain =3.85)	Average ^† † Average score per domain; =13.76 (SD ^‡ ‡ SD = Standard deviation per domain =3.57)	0.878	0.97
	Novice to Advanced Beginner	9.1% (3)	9.6% (5)
	Competent	48.5% (16)	61.5% (32)
	Proficient to Expert	42.4% (14)	28.9% (15)
				Total=0.980	-

[1] Corresponding author:
Daniel Gonçalves Campos E-mail: dangcampos@gmail.com https://orcid.org/0000-0003-0729-6441

Brasil

Brasil

Cultural adaptation and evaluation of the measurement properties of the Facilitator Competency Rubric for clinical simulation facilitators

Objective:

Method:

Results:

Conclusion:

Highlights:

Item	Synthesis version	Version resulting from the evaluation of the committee of experts
Concepts	Novice (1) to experienced beginner (2)	Novice (1) to advanced beginner (2)
1.1 Domain 1- Programming	- Can identify the need for small groups at the bedside. - Has creativity in planning activities. - Can program the learning experience in such a way that it is optimal.	- Identifies the need for small groups at the bedside. - Demonstrates creativity in planning activities. - Schedules the learning experience in such a way that it is optimal.
1.2 Domain 1 - Learning objectives	- Covers the cognitive, affective and psychomotor learning domains. - Correlates the objectives of all domains to the education or experience of the participants.	- Addresses the cognitive, affective and psychomotor learning domains. - Correlates the objectives of all domains to the educational level or experience of the participants.
1.3 Domain 1 - Planning process	- Informs laboratory team that a simulation will be conducted. - Works together with the laboratory team to fully achieve learning objectives. - Analyzes previous clinical simulations, aiming to improve the learning experience.	- Informs laboratory team of plans to conduct the simulation. - Collaborates with the laboratory team to fully achieve learning objectives. - Reviews previous clinical simulations to enhance the learning experience.
1.4 Domain 1 - Fidelity level (e.g., environment, simulation modality)	- Plans a level of fidelity that encompasses the desired outcomes.	- Plans the level of fidelity that encompasses the desired results.
1.5 Domain 1 - Availability of supplies/equipment	- Lists the supplies and equipment needed in the simulation - Organizes necessary learning materials by priority. - Develops or improves materials to stimulate critical thinking in learners.	- Lists the supplies and equipment needed for the simulation. - Organizes teaching materials by priority. - Develops or improves materials to stimulate critical thinking in participants.
1.6 Domain 1 - Preparation requirements	- Informs participants of what is necessary to prepare before the simulation. - Can assess whether participants are prepared for the simulation.	- Informs participants about what is necessary to prepare before the simulation. - Determines whether participants are prepared for the simulation.
1.7 Domain 1 - Assessment methods	- Tries to use good psychometric assessment tools.	- Plans to use assessment tools with good psychometric properties.
1.3 Domain 2 - Role identification	- Analyzes which role should be given to which participant to optimize learning, considering the strengths and weaknesses already identified.	- Analyzes which role should be given to each participant to optimize learning, considering the strengths and weaknesses already identified.
1.3 Domain 3 - Participants involvement	- During the simulation, gives good instructions or incentives for all participants to get involved. - Employs different methods to engage those who are not participating much.	- Provides clues or incentives for all participants to engage during the simulation. - Employs different methods to engage those who are not participating much in the simulation.
1.5 Domain 3 - Time/duration	- Continues the written scenario, regardless of time management. - Adapts throughout the experience, seeking to cover all learning objectives within the allocated time.	- Continues the scenario according to the script, without worrying about time management. - Adapts throughout the experience, seeking to cover all learning objectives within the established time.
1.2 Domain 4 - Facilitates reflection	- Stimulates in-depth analysis of choice processes and higher-order thinking.	- Stimulates in-depth analysis of decision-making processes, critical thinking and problem solving.
1.4 Domain 4 - Active listening	- In the discussion, contributes more than the participants themselves.	- Contributes more than the participants themselves during the discussion.
1.6 Domain 4 - Learning Objectives	- Pays attention to the events in the scenario. - Can assess whether the learning objectives were assimilated. - Assists participants in assessing the level of assimilation of learning objectives.	- Focus their attention on the events in the scenario. - Can assess whether the learning objectives were achieved. - Helps participants determine the achievement of learning objectives.
1.8 Domain 4 - Summary	- Helps participants to summarize the simulation.	- Supports participants as they summarize the simulation.
1.1 Domain 5 - Experience	- Uses designed methods to collect data from participants, team, and teaching staff about the simulation.	- Uses structured methods to collect data from participants, team, and teaching staff about the simulation.
1.2 Domain 5 - Participants	- Uses designed methods to collect data about participants and learning.	- Uses structured methods to collect data about participants and learning.