A small group learning model for evidence-based medicine

Background

Evidence-based medicine (EBM) is defined as “the conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of individual patients.”¹ Although the concepts of EBM have been discussed in medical education for the past 20 years, many medical learners and practicing physicians still lack the knowledge and skills needed to incorporate EBM into their patient care.² Lack of time, of skills, and knowledge, inadequate access to resources, and cost are identified as common barriers to learning and practicing EBM.³

While the principles and benefits of EBM are well-established, there is a relatively small body of literature addressing EBM instruction in medical education.⁴ A few studies have explored instructional methods, from teaching in clinical settings to seminars, workshops, and short courses.^5,6 Others looked at pedagogical efficacy by comparing in-person and online teaching, discussion-based and lecture formats, and single-discipline and interprofessional groups.⁴ Many of these studies, however, had methodological or outcome evaluation limitations.^5–7

Recently, scholars have called for educational approaches to EBM that are authentic, engaging, and student-centered.⁸ Small-group learning (SGL), sometimes called cooperative learning, has been used in secondary and higher education for many decades to promote student engagement.^9–12 SGL has been successfully adopted in medical education to enhance skills and knowledge in a variety of areas, including interpreting and applying evidence in clinical practice.^13–18 SGL utilizes an expert facilitator to guide face-to-face educational experiences and support learners to take active roles in their training.^8,14–16,19

The purpose of this study is to evaluate the effectiveness of SGL in teaching the fundamental EBM skills.

Methods

This study was set at a university-affiliated family medicine residency program located in a large Midwestern city. In the three academic years between 2013 and 2015, 40 incoming interns participated in the educational intervention. The workshop series, referred to as “EBM bootcamp”, was integrated into the Introduction to Family Medicine rotation, held each year during the interns’ first month in residency.

The workshop

EBM bootcamp was divided into four 2-hour sessions spread across the 4-week rotation. Residents worked in small groups of three to four to explore the fundamentals of EBM through interactive dialogue and mock clinical scenario practice. The overarching educational goal of the training was to identify clinically relevant questions and then formulate evidence-based answers. At the end of the training, learners were expected to be able to: 1) ask clinically relevant answerable questions, 2) find the evidence by searching the medical literature, 3) assess the quality of the evidence, 4) summarize medical articles, and 5) synthesize a conclusion to present an evidence-based answer. Details of training objectives and activities will be provided by the authors upon request. The principal investigator, who was also the instructor in the workshop, completed the grading of the learners’ assessments. The Indiana University Institutional Review Board reviewed and approved this study which was exempted from participant consent as the study reports on educational activity.

Study design

The intervention’s effectiveness was evaluated using pre- and post-assessments. At the start of the workshop, participants reported their prior experiences with EBM in medical school, their evidence-based resource use in clinical practice, their literature-reading habits, and their prior experience presenting in journal clubs or similar activities.

Assessments included both subjective and objective components. Interns completed a pre-test self-assessment and an EBM skills test at the start of the workshop series and again 4 weeks after completing the workshop. The self-assessment tool explored three primary domains of EBM: 1) formulating clinical questions, 2) searching the literature, and 3) identifying appropriate study designs. On the EBM skills test, the residents were presented with a clinical scenario and were asked to formulate a question, identify and summarize an article that addresses the question, and finally, explain their choice of best study design. Attitudes toward EBM were also assessed before and after the workshop. Besides this intervention, no further EBM training was taught between the completion of the course and the post test. The study was reviewed and deemed exempt by the university’s Institutional Review Board. The assessment tools, adapted from the original Fresno test,^20,21 are available in Supplementary materials. Supplementary materials contain the rubric used to evaluate resident responses. We adapted the Fresno test since some of the skills assessed by the original test (such as validity, significance, etc) were beyond the objectives of this training.

Statistical analysis

Performance on the EBM exercise was analyzed using Student’s t-test to compare each item before and after the training. Self-efficacy for using patient, intervention, comparison, and outcomes,²² identifying resources, and summarizing articles were evaluated on Likert scales. A self-efficacy score was generated from the summation of the Likert scale ratings for these three items. We used Student’s t-test to compare before and after scores.

We used regression analysis to examine the correlation between the self-assessment and EBM tests scores (dependent variables) and the residents’ characteristics, prior experiences, and baseline attitudinal indicators (independent variables). In addition, we used regression analysis to explain the improvement of scores using multiple models. We used a P-value of 0.05 as a cutoff to determine statistical significance. STATA 14 was used for the quantitative analysis.

Results

A total of 40 residents out of a potential 43 (93.0%) participated in the EBM bootcamps over the 3 years of the study. The three interns excluded from the study started residency 2–3 months late and thus could not participate in the training. The final analysis included all residents except in the case of the post-workshop assessment, for which two tests were lost. The baseline characteristics of the participants, the self-assessment scores, and the EBM skills test scores are presented in Table 1. When examining predictors of baseline performance, the practice of reviewing journals weekly was significantly associated with the EBM test score (P=0.02).

Table 1 Participants’ characteristics and baseline mean scores on the self-efficacy and the EBM test Note: *P<0.05. Abbreviations: EBM, evidence-based medicine; SD, standard deviation.

Individual self-efficacy

There was a significant improvement of 3.28 points (P<0.05) in the average self-assessed EBM skills score (from 9.66 to 12.95, out of a maximum score of 15). Improvement in the self-assessment score was inversely correlated with baseline self-assessment: the lower the baseline scores, the higher the improvement (P<0.05) (Figure 1).

Figure 1 Improvement in self-assessment as a function of baseline self-assessment score.

EBM skills assessment

There was a significant improvement of 1.68 points (P<0.05) in the average EBM skills test score (from 6.02 to 7.71, out of a maximum score of 9). Similar to the self-assessment scores, improvement on the EBM skills test was inversely correlated with baseline scores (P<0.05) (Figure 2). There was no correlation between subjective and objective score improvement (r=0.06, P=0.73).

Figure 2 EBM skills improvement as a function of baseline score. Abbreviation: EBM, evidence-based medicine.

Attitude toward EBM

All residents (100%) agreed or strongly agreed that EBM is important for a physician’s clinical practice. This view did not change after the training.

Resident evaluation of the workshop

The vast majority of residents (97.37%) agreed or strongly agreed that the EBM bootcamp was helpful in improving their EBM skills. Similarly, a majority of residents (86.84%) agreed or strongly agreed that the workshop series should always be a component of the orientation. The majority of residents (84.21%) said they would most likely use the skills learned during the series in their daily practice.

Discussion

The small-group EBM workshop yielded improvements in both EBM test scores and perception of skills. The workshop was highly acceptable to the residents. Residents with the lowest initial scores appeared to gain the most benefit from the intervention. The fact that poor initial performers on both tests improved more than their peers suggests that this brief intervention may help get everyone “up to speed” with EBM skills at the start of residency.

Our study is consistent with previous research examining the impact of focused training on EBM skills and knowledge.^4,23 For example, Dinkevich et al found a marked improvement in pediatrics interns’ EBM skills after 3 or 4 weekly seminars.²³ In their literature review, Hecht et al documented the effectiveness of various methods for learning EBM such as journal clubs, conferences, workshops, and courses.⁵ However, many such studies are undermined by significant potential bias.⁴ While our study does not compare different approaches, it is unique in documenting the effectiveness of one important learning method. The SGL model employed in this workshop series has many advantages for residents beyond the mastery of EBM basic skills. SGL facilitates working in teams, an essential skill for today’s residents and practicing physicians.¹⁹

Although our study was conducted at a single center and may thus not be generalizable to all other settings, including all the residents of three cohorts lend substance to the conclusion. Our study used a pre-and post-design, which does not provide strong evidence for causal inference between the intervention and the outcomes. However, the short period of time (4 weeks only) between the pre- and post-assessments and the absence of any other exposure to EBM concepts during this time support the validity of the findings. Future studies should use a multi-center experimental design to compare small-group learning to more commonly used methods such as didactic teaching to determine the best option in EBM education.

Conclusion

A small-group workshop yields improvements in skills and self-efficacy in EBM for starting residents.

Acknowledgment

This work would not have been possible without funding from the IU Health Value Grant (VFE-319).

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLoS One. 2014;9(1):e86706.
2.		Maggio LA, Tannery NH, Chen HC, ten Cate O, O’Brien B. Evidence-based medicine training in undergraduate medical education: a review and critique of the literature published 2006–2011. Acad Med. 2013;88(7):1022–1028.
3.		Sadeghi-Bazargani H, Tabrizi JS, Azami-Aghdash S. Barriers to evidence-based medicine: a systematic review. J Eval Clin Pract. 2014;20(6):793–802.
4.		Ilic D, Maloney S. Methods of teaching medical trainees evidence-based medicine: a systematic review. Med Educ. 2014;48(2):124–135.
5.		Hecht L, Buhse S, Meyer G. Effectiveness of training in evidence-based medicine skills for healthcare professionals: a systematic review. BMC Med Educ. 2016;16(1):103.
6.		Ahmadi SF, Baradaran HR, Ahmadi E. Effectiveness of teaching evidence-based medicine to undergraduate medical students: a BEME systematic review. Med Teach. 2014;37(1):21–30.
7.		Cheng HM, Guo FR, Hsu TF, et al. Two strategies to intensify evidence-based medicine education of undergraduate students: a randomized controlled trial. Ann Acad Med Singapore. 2012;41(1):4–11.
8.		Zee M, de Boer M, Jaarsma AD. Acquiring evidence-based medicine and research skills in the undergraduate medical curriculum: three different didactical formats compared. Perspect Med Educ. 2014;3(5):357–370.
9.		Webb NM. Peer interaction and learning in small groups. Int J Educ Res. 1989;13(1):21–39.
10.		Springer L, Stanne ME, Donovan SS. Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: a meta-analysis. Rev Educ Res. 1999;69(1):21–51.
11.		Towns MH, Kreke K, Fields A. An action research project: student perspectives on small-group learning in chemistry. J Chem Educ. 2000;77(1):111.
12.		van Blankenstein FM, Dolmans DHJM, van der Vleuten CPM, Schmidt HG. Which cognitive processes support learning during small-group discussion? The role of providing explanations and listening to others. Instr Sci. 2011;39(2):189–204.
13.		Rial J, Scallan S. Practice-based small group learning (PBSGL) for CPD: a pilot with general practice trainees to support the transition to independent practice. Educ Prim Care. 2013;24(3):173–177.
14.		Durning SJ, Conrad RM. Small-group teaching. In: Dent JA, Harden RM, editors. A Practical Guide for Medical Teachers. 4th ed. London: Elsevier; 2013:69–74.
15.		Harden RM, Laidlaw JM. Essential Skills for a Medical Teacher: An Introduction to Teaching and Learning in Medicine. London: Elsevier; 2012:137–143.
16.		McCrorie P. Teaching and leading small groups. In: Swanwick T, editor. Understanding Medical Education: Evidence, Theory, and Practice. 2nd ed. Chichester, West Sussex: Wiley Blackwell; 2014:123–136.
17.		Peloso PM, Stakiw KJ. Small-group format for continuing medical education: a report from the field. J Contin Educ Health Prof. 2000;20(1):27–32.
18.		Thomas KG, Thomas MR, York EB, Dupras DM, Schultz HJ, Kolars JC. Teaching evidence-based medicine to internal medicine residents: the efficacy of conferences versus small-group discussion. Teach Learn Med. 2005;17(2):130–135.
19.		Jackson D, Hickman LD, Power T, Disler R, Potgieter I, Deek H, Davidson PM. Small group learning: graduate health students’ views of challenges and benefits. Contemp Nurse. 2014;48(1):117–128.
20.		Ramos KD, Schafer S, Tracz SM. Validation of the Fresno Test of competence in evidence based medicine. BMJ. 2003;326(7384):319–321.
21.		McCluskey A, Bishop B. The adapted Fresno Test of competence in evidence-based practice. J Contin Educ Health Prof. 2009;29(2):119–126.
22.		Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995;123(3):A12–A13.
23.		Dinkevich E, Markinson A, Ahsan S, Lawrence B. Effect of a brief intervention on evidence-based medicine skills of pediatric residents. BMC Med Educ. 2006;6:1.