Mapping the scientific research on the negative aspects of the medical school learning environment

SUMMARY

Objective:

We sought to understand the landscape of published articles regarding medical schools’ learning environments (LE) worldwide, with an explicit focus on potentially negative aspects of the LE as an effort to identify areas specifically in need of remediation or intervention that could prevent future unprofessional behaviours, burnout, violence and mistreatment among students and physicians.

Methods:

A bibliometric analysis was conducted in six electronic databases (PubMed/Medline, Web of Science, Cochrane Library, SCOPUS, ERIC-ProQuest and PsycINFO) through December 31, 2016, including 12 themes: learning environment - general, hidden curriculum (negative), unethical behaviours, bullying/hazing, violence, sexual discrimination, homophobia, racism, social discrimination, minorities’ discrimination, professional misconduct, and "other" negative aspects.

Results:

Of 9,338 articles found, 710 met the inclusion criteria. The most common themes were general LE (233 articles), unprofessional behaviours (91 articles), and sexual discrimination (80 articles). Approximately 80% of articles were published in the 21st century.

Conclusion:

There is a clear increase in scientific articles on negative aspects of the medical school LE in high-quality journals, especially in the 21st century. However, more studies are needed to investigate negative LE aspects with greater attention paid to experimental, longitudinal, and cross-cultural study designs.

KEYWORDS:
Learning Environment; Medical Education; Medical Students; Ethics; Professionalism

RESUMO

OBJETIVO:

Buscou-se entender o panorama dos artigos publicados sobre os ambientes de aprendizagem (AA) das escolas médicas em todo o mundo, com um foco explícito nos aspectos potencialmente negativos do AA como um esforço para identificar áreas específicamente necessitadas de remediação ou intervenção que poderiam evitar futuros comportamentos não profissionais, violência e maus-tratos entre estudantes e médicos.

Métodos:

Foi realizada uma análise bibliométrica em seis bases de dados eletrônicas (PubMed/Medline, Web of Science, Biblioteca Cochrane, Scopus, Eric-ProQuest e PsycInfo) até 31 de dezembro de 2016, incluindo 12 temas: ambiente de aprendizagem - geral, currículo oculto (negativo), comportamentos antiéticos, bullying/trote, violência, discriminação sexual, homofobia, racismo, discriminação social, discriminação de minorias, má conduta profissional e “outros" aspectos negativos.

Resultados:

Dos 9.338 artigos encontrados, 710 preencheram os critérios de inclusão. Os temas mais comuns foram LE geral (233 artigos), comportamentos não profissionais (91 artigos) e discriminação sexual (80 artigos). Aproximadamente 80% dos artigos foram publicados no século XXI.

Conclusão:

Há um claro aumento em artigos científicos sobre aspectos negativos da escola de medicina LE em periódicos de alta qualidade, especialmente no século XXI. No entanto, mais estudos são necessários para investigar aspectos negativos do LE com maior atenção aos desenhos de estudos experimentais, longitudinais e transculturais.

PALAVRAS-CHAVE:
Ambiente de aprendizagem; Educação médica; Estudantes de medicina; Ética; Profissionalismo

INTRODUCTION

Environment can be defined as “the surroundings or conditions in which a person, animal, or plant lives or operates”.1 This concept, derived from the biological sciences, has increasingly been the subject of study in medical sciences and educational research. Specifically for medical schools, a student's “surroundings or conditions” encompass physical, social, and psychological influences and must be conducive to developing the knowledge, attitudes, skills, and behaviours students will need to practice as physicians. The terminology used to describe this environment varies, and has included educational environment,2 teaching environment,3 and, most commonly, the learning environment.4

The first study that focused on studying learning environments (LE) in higher education dates back to 1958.5 Concerning medical schools' LEs, Hutchins (in 1961)6 developed the first questionnaire and the first attempt to understand quantitatively how the environment might impact students' attitudes, values, and behaviours. Since then, many tools have been developed to assess medical students' perceptions of their LE,7 showing the growing importance of this subject to medical researchers. According to Cohen,8 if medical schools intend to deal with the erosion of professionalism during the course of medical training, “purging their own learning environments of unprofessional practices” (p. 610) is a key endeavour.

Much research now supports Cohen's8 idea that this purging of negative aspects of the LE is critical for developing a professional physician. Much of this focuses on the cognitive/curricular or social aspects of the LE and was derived from small populations of students.9 Reported perceptions of poor LE have already been correlated with high levels of student burnout and worse perception of quality of life,10 decreased personal growth,11 worse academic performance on the United States Medical Licensing Examination Step 1,12 and also less time spent by students on activities involving direct patient contact.13

However, many LE general instruments used in research might not encompass all aspects of the LE that influence students' lives. Specifically, it may be especially important to consider the negative aspects of the LE because these are likely to have ensuing negative effects on medical students, as has been demonstrated for unprofessional/unethical behaviours, violence, and harassment, and their impact on students' professionalism and quality of life.14^-18

Mapping the research on the negative aspects of the LE will help to identify areas that are already well explored, areas where more work needs to be done, how interests have trended over time among researchers, and the geographic and cultural areas where interest in these topics is greatest. Thus, the purpose of this study was to build on our current understanding of LEs through a comprehensive bibliometric analysis that develops a broader framework for LEs and their negative aspects, which can guide further investigation about each specific topic (such as original studies and systematic reviews) and medical curricula interventions.

METHODS

From September 2016 to January 2017, we carried out a bibliometric analysis to evaluate all original articles related to medical schools' LE up to December 31^st, 2016. This bibliometric approach is defined as “a tool by which the state of science and technology can be observed through the overall production of scientific literature”19 (p. 6) and is used to map a field of research, providing a statistical description of this (recent and/or historical) data.19^-21

Since this is a review of literature, ethical approval is not required for this project. The sequence of the main phases is described below.

KEYWORDS SELECTION

Initially, three authors (R.F.D., A.O.C., J.G.O.) reviewed a sample of LE articles and independently generated a list of keywords focusing on capturing all potentially negative aspects of medical school LEs. After each list was created, four authors (R.F.D., A.O.C., J.G.O., G.L.) examined the list to eliminate redundancies and add new words to the list. Then, each independent researcher (R.F.D., A.O.C., J.G.O.) created a list of themes (clusters) made up of related keywords and compared them with the Learning Environment literature to check for alignment with the current scientific data. For this initial stage of development of themes, we examined systematic reviews and the most prominent articles published in high tier journals.

One author (R.F.D.), in collaboration with the institution librarian, was responsible for comparing each list of themes, merging similar ones and removing duplicates. A discussion among all authors brought a consensus of twelve thematic clusters, including the “general LE” and eleven negative aspects of medical schools' LE: hidden curriculum (negative), unethical behaviours, bullying/hazing, violence, sexual discrimination, homophobia, racism, social discrimination, minorities discrimination, professional misconduct, and other negative aspects. These clusters were arbitrarily defined by the authors of this paper. The general theme was based on articles with a focus on the Learning Environment, usually containing “Learning Environment” in the title and using LE measurement instruments. However, the negative aspects included research that did not always define the negative aspect as a component of the LE. Figure 1 summarizes the conceptual framework of the LE used by this manuscript.

FIGURE 1
CONCEPTUAL FRAMEWORK OF LE USED IN THIS ARTICLE.

ELIGIBILITY CRITERIA

Inclusion criteria were: original studies (longitudinal studies - cohort and case-control, cross-sectional studies, case reports, experience reports and experimental studies) carried out with medical students and related to their LE. Studies considered out of area (not related to LE) and out of population (not with medical students) were excluded. Furthermore, as our focus was on original studies, reviews, replications, theoretical pieces, articles without abstracts (because we needed the abstracts in order to review the paper), and book chapters were not included. No language limit was applied.

Databases search

We conducted a search in six electronic databases (PubMed/Medline, Web of Science, Cochrane Library, ERIC-ProQuest, SCOPUS, and PsycINFO) including all studies published up to December 31, 2016. A variety of Boolean expressions based on the twelve thematic clusters were created to guide the search in these databases (see Supplemental Material 1), and then each database outcome was exported to Mendeley Desktop version 1.17.6 (a free reference management program - ELSEVIER®) and sorted alphabetically in order to facilitate the review process. A hand search of references from the oldest articles identified one additional article6 that was included in our analysis.

Data collection

Three reviewers (R.F.D., A.O.C., J.G.O.) independently screened the title, authors, and abstracts to determine if they met inclusion and exclusion criteria. If excluded, the reason for exclusion as described in the previous section was noted, and if included, the classification into one of the twelve clusters as defined above was noted. Papers that mentioned more than one theme mentioned above were discussed by all authors, who came to consensus on the most relevant finding of the article. Any discrepancies were resolved by a discussion among the reviewers in a follow-up meeting. The intra-class correlation between reviewers was assessed for the first 100 studies, based on the choice of inclusion/exclusion criteria, and we found an intra-class correlation coefficient of 0.915, showing excellent reliability between the three reviewers.

Bibliometric analysis

All included articles were exported to Excel for Mac version 14.7.2 (Microsoft^®), and then each article was classified according to its characteristics: title, authors' name, journal title, journal's impact factor (by Web of Science, 2015), study design, year of publication, article's number of citations (by Web of Science and Google Scholar), and country of origin (of the corresponding author). Then descriptive statistics of all variables were analysed.

RESULTS

We found 9,337 articles across 6 databases and 1 article via hand search, resulting in a total of 9,338 articles (see Supplemental Material 2). After dropping duplicates (using the automatic Mendeley function), 5,155 articles remained. Based on our eligibility criteria, 4,445 articles were withdrawn due to one of the following reasons: no abstract (416 articles); book chapter (20 articles); duplicate (missed by Mendeley; 376 articles); review (123 articles); theoretical articles (467 articles); out of population (not on medical students; 567 articles); and out of area (not on LE; 2,476 articles).

Finally, 710 articles were included in this bibliometric analysis. Each article was classified into one of the twelve themes, resulting in a final distribution as follows: learning environment - general (233 articles – 32.8%); unprofessional behaviours (91 articles – 12.8%); sexual discrimination (80 articles – 11.3%); minorities discrimination (76 articles – 10.7%); violence (65 articles – 9.1%); hidden curriculum (52 articles – 7.3%); unethical behaviours (49 articles – 6.9%); racism (16 articles – 2.2%); homophobia (13 articles – 1.8%); bullying/hazing (7 articles – 1.0%); social discrimination (5 articles – 0.7%); and other (23 articles – 3.2%).

Supplemental Material 3 shows the distribution of publications of all articles related to medical schools' LE included in this manuscript. The first such publication dates back to 1961;6 publications remained relatively stable and infrequent until the 21^st century, when there was a notable increase in manuscripts related to medical schools' LE. In fact, 80% of all articles related to medical schools' LE were published after 1999. Notably, a large spike occurs at approximately 2006; almost 70% of all articles were published from 2006 through 2016.

The characteristics of these studies are shown in Table 1. Most of them (77.6%) are cross-sectional and quantitative studies (67.4%). We found only 41 (5.8%) experimental studies and 75 (10.5%) longitudinal studies. Almost two-thirds of the articles were published in journals with an impact factor (IF) greater than 1.00, with one third published in journals with IF greater than 3.00. Academic Medicine (IF 4.194), Medical Education (IF 3.369), Medical Teacher (IF 2.355), BMC Medical Education (IF 1.312) and Teaching and Learning in Medicine (IF 1.159) represent five leading journals in this area. Most articles on this topic had corresponding authors who resided in the United States (43.5%), followed by the United Kingdom (8.6%) and Canada (5.5%).

TABLE 1
CHARACTERISTICS OF STUDIES ON MEDICAL SCHOOLS’ LEARNING ENVIRONMENT

All Studies (N = 710)
Characteristics		No. Studies
Publication year
	Until 1980	15 (2.1%)
	1981-1990	32 (4.4%)
	1991-2000	97 (13.7%)
	2001-2010	236 (33.2%)
	2011-2016	330 (46.5%)
Study Design
	Cross-Sectional	551 (77.6%)
	Longitudinal	75 (10.6%)
	Experimental	41 (5.8%)
	Experience Report	39 (5.5%)
	Case Report	4 (0.6%)
Measurement Methods (only if cross-sectional or longitudinal)
	Quantitative	422 (67.4%)
	Qualitative	160 (25.6%)
	Qualitative / Quantitative	44 (7.0%)
Journals
	Academic Medicine	126 (18.4%)
	Medical Education	67 (9.8%)
	Medical Teacher	48 (7.0%)
	BMC Medical Education	29 (4.2%)
	Teaching and Learning in Medicine	17 (2.5%)
Impact factor journals (WoS)
	No impact factor	188 (26.5%)
	0.000 – 1.00	65 (9.2%)
	1.01 – 3.00	219 (30.8%)
	> 3.00	237 (33.4%)
Countries (by author's affiliation)
	United States	309 (43.5%)
	United Kingdom	61 (8.6%)
	Canada	39 (5.5%)
	Australia	32 (4.5%)
	Netherlands	23 (3.2%)

Finally, Table 2 presents the 20 most-cited articles in the area of medical schools' LE. The number of citations for these articles is quite high, with 6 articles having more than 100 citations in Web of Science (WoS) and 10 articles having more than 200 citations in Google Scholar.

TABLE 2
MOST CITED ARTICLES ON MEDICAL SCHOOLS’ LEARNING ENVIRONMENT

Rank	Article	No. WoS Citations	No. Google Scholar Citations	Average Citations/ Year - WoS
1	Lempp H, Seale C. The hidden curriculum in undergraduate medical education: qualitative study of medical students' perceptions of teaching. BMJ. 2004;329(7469):770-3.	190	471	15.8
2	Sheehan KH, Sheehan DV, White K, Leibowitz A, Baldwin DC Jr. A pilot study of medical student ‘abuse'. Student perceptions of mistreatment and misconduct in medical school. JAMA. 1990;263(4):533-7.	190	311	7.3
3	Papadakis MA, Hodgson CS, Teherani A, Kohatsu ND. Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med. 2004;79(3):244-9.	189	358	15.7
4	Christakis DA, Feudtner C. Ethics in a short white coat: the ethical dilemmas that medical students confront. Acad Med. 1993;68(4):249-54.	151	248	6.6
5	Dyrbye LN, Massie FS Jr, Eacker A, Harper W, Power D, Durning SJ, Thomas MR, Moutier C, Satele D, Sloan J, Shanafelt TD. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA. 2010;304(11):1173-80.	149	318	24.8
6	Richman JA, Flaherty JA, Rospenda KM, Christensen ML. Mental health consequences and correlates of reported medical student abuse. JAMA. 1992;267(5):692-4.	122	236	5.1
7	Kassebaum DG, Cutler ER. On the culture of student abuse in medical school. Acad Med. 1998;73(11):1149-58.	97	205	5.4
8	Moffat KJ, McConnachie A, Ross S, Morrison JM. First-year medical student stress and coping in a problem-based learning medical curriculum. Med Educ. 2004;38(5):482-91.	95	298	7.9
9	Karnieli-Miller O, Vu TR, Holtman MC, Clyman SG, Inui TS. Medical students' professionalism narratives: a window on the informal and hidden curriculum. Acad Med. 2010;85(1):124-33.	90	154	15
10	Baxter N, Cohen R, McLeod R. The impact of gender on the choice of surgery as a career. Am J Surg. 1996;172(4):373-6.	87	136	4.3
11	Madigosky WS, Headrick LA, Nelson K, Cox KR, Anderson T. Changing and sustaining medical students' knowledge, skills, and attitudes about patient safety and medical fallibility. Acad Med. 2006;81(1):94-101.	85	167	8.5
12	Papadakis MA, Osborn EH, Cooke M, Healy K. A strategy for the detection and evaluation of unprofessional behavior in medical students. University of California, San Francisco School of Medicine Clinical Clerkships Operation Committee. Acad Med. 1999;74(9):980-90.	85	127	5
13	Patenaude J, Niyonsenga T, Fafard D. Changes in students' moral development during medical school: a cohort study. CMAJ. 2003; 168(7): 840-844.	83	223	6.4
14	Hicks LK, Lin Y, Robertson DW, Robinson DL, Woodrow SI. Understanding the clinical dilemmas that shape medical students' ethical development: questionnaire survey and focus group study. BMJ. 2001; 322(7288): 709-710.	76	207	5.1
15	Roberts LW, Warner TD, Lyketsos C, Frank E, Ganzini L, Carter D. Perceptions of academic vulnerability associated with personal illness: a study of 1,027 students at nine medical schools. Collaborative Research Group on Medical Student Health. Compr Psychiatry. 2001;42(1):1-15.	74	135	4.9
16	Dyrbye LN, Thomas MR, Harper W, Massie FS Jr, Power DV, Eacker A, Szydlo DW, Novotny PJ, Sloan JA, Shanafelt TD. The learning environment and medical student burnout: a multicentre study. Med Educ. 2009;43(3):274-82.	70	153	10
17	Edwards MT, Zimet CN. Problems and concerns among medical students--1975. J Med Educ. 1976;51(8):619-25.	68	98	1.7
18	Stern DT, Frohna AZ, Gruppen LD. The prediction of professional behaviour. Med Educ. 2005;39(1):75-82.	66	130	6
19	Frank E, Carrera JS, Stratton T, Bickel J, Nora LM. Experiences of belittlement and harassment and their correlates among medical students in the United States: longitudinal survey. BMJ. 2006;333(7570):682.	64	162	6.4
20	Wolf TM, Randall HM, von Almen K, Tynes LL. Perceived mistreatment and attitude change by graduating medical students: a retrospective study. Med Educ. 1991;25(3):182-90.	63	116	2.5

Supplemental Materials 4-7 present the characteristics of each of the 12 themes defined by this article. The journal Academic Medicine has published the most articles on medical students' LE across most of the dimensions we examined. Whereas researchers from the United States published most of the papers in this field, Pakistan published the most in bullying/ hazing. Finally, some areas are quite new in the scientific literature, and until the 1980s there were no published articles on the following areas: unprofessional behaviours, violence, hidden curriculum (negative), unethical behaviour, racism, homophobia, and bullying/hazing.

In relation to the instruments used to measure the LE, we found high use of the following tools: the Dundee Ready Education Environment Measure (DREEM),2 the Medical Student Learning Environment Scale (MSLES),22^,23 and the most recent Johns Hopkins Learning Environment Scale (JHLES).4 Comparing articles using these tools (cluster: learning environment - general) with all articles, we found an even higher proportion of manuscripts published between 2011-2016 (53%) that used these popular measurement tools.

DISCUSSION

This study represents the first comprehensive bibliometric description for learning environment (LE) research in medical schools with a focus on negative aspects. We identify here the relatively recent rapid increase in interest in LE globally, demonstrating increased awareness of the importance of this topic, and also the recent attempts to improve LE research study designs. For educators, this is an important call to increase our attention to the importance of the medical school LE as well as to explore in greater depth the potentially negative aspects of medical school LEs.

To facilitate our understanding about the conceptual framework of the LE area, we developed a concept map (Figure 1) based on the authors' own experience and the most prominent studies reviewed by this manuscript. First, our understanding is that both positive and negative aspects might influence students' perception of their LE. In this article, we decided to focus more on the negative ones. We have identified ten main negative areas that might impact students' perception of their LE (in parentheses the most cited on each area): unprofessional behaviors;17 sexual discrimination;24 minorities discrimination;25 violence;26 hidden curriculum (negative);27 unethical behaviors;28 racism;28 homophobia;30 bullying/ hazing;31 and social discrimination.32 These negative themes may be concomitantly the source and the consequence of a “bad” LE; that is, these negative environments may create a poor perception that creates a positive feedback on these unacceptable behaviours. Educators should be aware of this, attempting to prevent a state where the “mean” turns inherent and cannot be seen, such as in the famous study of the Stanford prison experiment.33

Noteworthy is that while there has been an increasing number of articles published in medical education in general, not all content areas receive the same attention.34 In our study, we found a recent and rapidly growing interest in medical school LE research, with the great majority (approx. 70%) of studies having been published from 2006 onward and in journals with an IF greater than 1.00, further indicating interest in this field among medical education researchers. What may be driving this interest in the medical school? First, our data suggest that the availability of LE measurement instruments may facilitate research. Over half of the published articles that utilized the most frequently used measures were published since 2011. Second, interest is likely being driven by greater concern for the potential negative impact of poor learning environments in medical schools, particularly student mistreatment and overwork, as these impact trainees' empathy and well-being.10^,34^-36 For example, in the U.S., the American Medical Association, driven by these concerns, began supporting a longitudinal study of medical students at 28 medical schools in 2010 that has already led to several publications.37^,38 Third, medical school accreditation standards may play a role, as the creation of a new Liaison Committee on Medical Education39 (LCME) standard related to LE quality coincides with the rapid increase in LE studies.

The United States has by far the most LE research identified in this study, followed by the United Kingdom, Canada, Australia, and the Netherlands. These findings are similar to analyses of all medical education articles40^,41 and global scientific production as a whole. Given that medical education research on the learning environment is dominated by several countries that do not represent most of the world's medical schools, educators and investigators must be cautious to ensure that they focus on local needs and do not seek to overgeneralize their results. For example, in the field of empathy research, a decline in student empathy during medical schools seems to have been largely driven by several studies from the U.S., and they were not corroborated by the bulk of international studies.42 At the same time, when reviewing the top countries by topic, we saw that some developing countries had the greatest number of publications for certain clusters. For example, Pakistan had the most publications on bullying/hazing and Nigeria had the third most in violence. Future studies should explore whether these findings reflect the initiative of individuals or research teams or a larger trend in negative LE aspects. Considering that, like most medical education research, cross-sectional and single site studies comprised the majority of LE research, future work should use multiple methods across cultures to better understand the complex interactions between students and their LEs.

Our findings have implications for health managers and medical educators, providing further evidence that the LE could have a positive but also a negative influence on medical students. Educators must promote appropriate infrastructure, active learning strategies, good clinical scenarios, high- and low-fidelity labs, formative feedback, valuable educational content, and consistent theoretical and practical assessments. Yet, on the other hand, educators must be aware that unethical behaviours, bullying, violence, sexual discrimination, professional misconduct, and the hidden curriculum could impair medical students' academic performance and mental health. Thus, they must become aware of these possible behaviours occurring in their curricula. The early identification of these negative aspects of the LE and the implementation of educational and preventive interventions should serve to minimize medical training distress and future unprofessional behaviours.

This research has several limitations. First, we focused only on original studies to characterize the current evidence base related to negative aspects of the LE, which meant that we excluded many other highly cited reviews and theoretical studies that may be influential in this field. Second, we may not have identified every relevant original research article, as no database has every paper and no search strategy can find every paper. However, we did not limit by language, so this is truly an international search. Third, the databases that we searched tended to be more focused on American and European journals, possibly excluding important contributions from the southern hemisphere. Fourth, this is a novel operationalization of the learning environment and has limitations and potential biases; however, it is important to initiate a discussion about all these possible important aspects of the LE. Fifth, many of the articles included more than one of the negative areas identified in this study. However, we tried to isolate the most important subject of each manuscript in order to facilitate our understanding. Finally, we focused more on negative aspects of the learning environment. Positive aspects, such as role models, peer relationships, and sense of engagement, clearly are important in understanding medical students' LE. However, we concentrated on negative aspects because the focus of this paper was to identify research that is related to negative outcomes for medical students and that has potential for remediation. In the future, it will be important to incorporate both negative and positive aspects of the medical school LE that contribute to changes in students' empathy and well-being.

CONCLUSION

Our analysis identifies the most important areas that articles, authors, and countries have studied or reported on in terms of negative aspects of the learning environment in medical schools. We demonstrated an important growth of scientific production in high-quality journals, especially in the 21^{st century}. However, more studies are needed that investigate the negative aspects of medical students' LE, with particular attention to experimental and cross-cultural/multi-school studies. Heightened awareness of negative aspects of the medical student LE should be useful in empowering medical professionals to make changes in the LE that will, in turn, improve student professionalism.

Acknowledgments

The authors wish to show our gratitude to Antonio P. de Melo Maricato for his contribution, and Dr. Robert B. Shochet and Dr. J. Kevin Dorsey for their kind review and support for this article.

32 REFERENCES

• 1. Oxford Living Dictionary. Environment Definition. Oxford Living Dictionary. https://en.oxforddictionaries.com/definition/environment Accessed July 10, 2017.
  » https://en.oxforddictionaries.com/definition/environment
• 2. Roff S, Mcaleer S, Harden RM, et al. Development and validation of the Dundee Ready Education Environment Measure (DREEM). Med Teach. 1997;19: 295-299.
• 3. Quaintance JL, Arnold L, Thompson GS. Development of an instrument to measure the climate of professionalism in a clinical teaching environment. Acad Med. 2008;83(10): S5-S8.
• 4. Shochet RB, Colbert-Getz JM, Wright SM. The John Hopkins learning environment scale: measuring medical students’ perceptions of the process supporting professional formation. Acad Med. 2015;90(6): 810-818.
• 5. Pace CR, Stern GG. An approach to the measurement of psychological characteristics of college environments. J Educ Psychol. 1958;40: 269-277.
• 6. Hutchins EB. The 1960 medical school graduate: his perception of his faculty, peers, and environment. J Med Educ. 1961;36: 322-329.
• 7. Colbert-Getz JM, Kim S, Goode VH, Shochet RB, Wright SM. Assessing medical students’ and residents’ perceptions of the learning environment: exploring validity evidence for the interpretation of scores from existing tools. Acad Med. 2014;89(12): 1687-93.
• 8. Cohen JJ. Professionalism in medical education, an American perspective: from evidence to accountability. Med Educ. 2006;40:607-617.
• 9. Wasson L, Cusmano A, Meli L, et al. Association between learning environment interventions and medical student well-being: a systematic review. JAMA. 2016;316(21): 2237-2252.
• 10. Tackett S, Wright S, Lubin R, Li J, Pan H. International study of medical school learning environments and their relationship with students well-being and empathy. Med Educ. 2017;51(3): 280-289.
• 11. Colbert-Getz J, Tackett S, Wright SM, Shochet RS. Does academic performance or personal growth share a stronger association with learning environment perception? Int J Med Educ. 2016;7: 274-278.
• 12. Wayne SJ, Fortner SA, Kitzes JA, Timm C, Kalishman S. Cause or effect? The relationship between student perception of the medical school learning environment and academic performance on USMLE Step 1. Med Teach 2013;35(5): 376-80.
• 13. van Hell EA, Kuks JB, Cohen-Schotanus J. Time spent on clerkship activities by students in relation to their perception of learning environment quality. Med Educ. 2009;43(7): 674-9.
• 14. Benbassat J. Undesirable features of the medical learning environment: a narrative review of the literature. Adv in Health Sci Educ. 2013;18: 527-536.
• 15. Fargen KM, Drolet BC, Philibert I. Unprofessional behaviours among tomorrow's physicians: review of the literature with a focus on risk factors, temporal trends, and future directions. Acad Med. 2016;91(6): 858-864.
• 16. Fnais N, Soobiah C, Chen MH, et al. Harassment and discrimination in medical training: a systematic review and meta-analysis. Acad Med 2014;89, 817-827.
• 17. Papadakis MA, Hodgson CS, Teherani A, Kohatsu ND. Unprofessional behaviour in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med 2004;79: 244-249.
• 18. Papadakis MA, Teherani A, Banach MA, et al. Disciplinary action by medical boards and prior behaviour in medical school. N Engl J Med. 2005;353: 2673-2682.
• 19. Okubo Y Bibliometric indicators and analysis of research systems: methods and examples. 1st ed. OECD Science, Technology and Industry Working Papers; 1997.
• 20. Azer SA. The top-cited articles in medical education: a bibliometric analysis. Acad Med. 2015;90(8): 1147-1161.
• 21. Broadus RN. Toward a definition of "bibliometrics". Scientometrics. 1987; 12:373-379.
• 22. Marshall RE. Measuring the medical school learning environment. J Med Educ 1978;53: 98-104.
• 23. Rosenbaum ME, Schwabbauer M, Kreiter C, Ferguson KJ. Medical students’ perceptions of emerging learning communities at one medical school. Acad Med. 2007;82(5): 508-15.
• 24. Baxter N, Cohen R, McLeod R. The impact of gender on the choice of surgery as a career. Am J Surg. 1996;172(4): 373-6.
• 25. Dyrbye LN, Thomas MR, Huschka MM, et al. A multicenter study of burnout, depression, and quality of life in minority and nonminority US medical students. Mayo Clinic Proceedings. 2006;11(81): 1435-1442.
• 26. Sheehan KH, Sheehan DV, White K, Leibowitz A, Baldwin DC Jr. A pilot study of medical student ‘abuse'. Student perceptions of mistreatment and misconduct in medical school. JAMA. 1990;263(4): 533-7.
• 27. Lempp H, Seale C. The hidden curriculum in undergraduate medical education: qualitative study of medical students’ perceptions of teaching. BMJ 2004;329(7469): 770-3.
• 28. Chrisdakis DA, Feudtner C. Ethics in a short white coat: the ethical dilemmas that medical students confront. Acad Med. 1993;68(4):249-54.
• 29. Dyrbye LN, Thomas MR, Eacker A, et al. Race, ethnicity, and medical student well-being in the United States. Archives of Internal Medicine. 2007;19(167): 2103-2109.
• 30. Klamen DL, Grossman LS, Kopacz DR. Medical student homophobia. Journal of Homosexuality. 1999;1(37): 53-63.
• 31. Ahmer S, Yousafzai AW, Bhutto N, Alam S, Sarangzai AK, Iqbal A. Bullying of medical students in Pakistan: a cross-sectional questionnaire survey. PLoS ONE. 2008;3(12): e3889. https://doi.org/10.1371/journal.pone.0003889
  » https://doi.org/10.1371/journal.pone.0003889
• 33. Zimbardo P. The Stanford Prison Experiment: A Simulation Study of the Psychology of Imprisonment. 1971. Stanford University, Stanford Digital Repository, Stanford.
• 34. Dyrbye LN, Thomas MR, Harper W, et al. The learning environment and medical student burnout: a multicentre study. Med Educ. 2009;43(3): 27482.
• 35. Dyrbye LN, Massie FS Jr, Eacker A, et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA 2010;304(11): 1173-80.
• 36. Neumann M, Edelhâuser F, Tauschel D, et al. Empathy decline and its reasons: a systematic review of studies with medical students and residents. Acad Med. 2011;86(8): 996-1009.
• 37. Skochelak S, Stansfield RB, Dunham L, et al. Medical student perceptions of the learning environment at the end of the first year: a 28-medical school collaborative. Acad Med. 2016;91(9): 1257-1262.
• 38. Smith SD, Dunham L, Dekhtyar M, et al. Medical student perception of the learning environment: learning communities are associated with a more positive learning environment in a multi-institutional medical school study. Acad Med. 2016;91(9): 1263-1269.
• 39. LCME - Liaison Committee on Medical Education. Accreditation standards. Washington, DC: Association of American Medical Colleges. 2016. http://lcme.org/publications/ Accessed April 11,2017.
  » http://lcme.org/publications/
• 40. Doja A, Horsley T, Sampson M. Productivity in medical education research: an examination of countries of origin. BMC Med Educ. 2014;14: 243. doi: 10.1186/s12909-014-0243-8.
  » https://doi.org/10.1186/s12909-014-0243-8
• 41. Tutarel O. Geographical distribution of publications in the field of medical education. BMC Med Educ. 2002;2:3. doi: 10.1186/1472-6920-2-3.
  » https://doi.org/10.1186/1472-6920-2-3
• 42. Ferreira-Valente A, Monteiro JS, Barbosa RM, Salgueira A, Costa P, Costa MJ. Clarifying changes in student empathy throughout medical school: a scoping review. Adv Health Sci Educ 2016;22(5): 1293-1313.

Publication Dates

• Publication in this collection
  Nov 2018

History

• Received
  07 Aug 2018
• Accepted
  27 Aug 2018