Evidence-Based Medical Education and Knowledge Translation

Conditions for Educational Evidence to Lead to Practice

Evidence-based medical education addresses how knowledge gained from research can inform educational content, teaching strategies, assessment, and curriculum delivery. Knowledge synthesis brings together findings from multiple studies to inform educational judgment. Knowledge translation (KT) addresses how that information is applied in light of learners’ levels, resources, and local operating conditions. The process through which evidence informs educational practice involves both providing applicable information and making continuous adjustments based on implementation experience.

Readiness of Knowledge Synthesis for Educational Practice Application

A study of the readiness of BEME Guides for application examined what information knowledge syntheses provide to educators (Maggio et al., 2018). The research team defined readiness in terms of quality, accessibility, and relevance. Guides were selected for analysis by comparing Web of Science search results from May 1, 2017, with the BEME website. The sample consisted of 42 guides published between 1999 and 2017, including 36 systematic reviews, 3 reviews, 2 realist reviews, and 1 review that graded the quality of papers. The guides included an average of 53 primary studies, with a range of 5–415. Two reviewers independently reviewed all guides, keeping narrative records for each guide and coding the 25 items in STORIES, which specifies minimum reporting requirements for knowledge synthesis in health professions education. Discrepancies were resolved by a third party. The agreed-upon data were shared among research team members with expertise in HPE, knowledge synthesis, knowledge translation, and information science.

All 42 guides included structured summaries, rationales for conducting the reviews, summaries of the included studies, and results presented in relation to the objectives, but none met all 25 items of STORIES. Fifteen guides reported PICO elements, and 4 guides included a meta-analysis. Heterogeneity in educational content and duration, participating professions, and study design was cited as the main reason for not conducting a meta-analysis. The BEME website provided 26 guides for free. As of March 24, 2018, journal websites offered 11 free open-access copies, with some overlap. The 11 guides that were not open access were all published between 2015 and 2017. Search strategies, extraction tools, and results tables were distributed across the two websites, neither of which provided the complete data. Of the guides, 35 provided practice points and 34 provided BEME Spotlight, but the PDFs did not indicate whether supplementary materials were available.

The research team examined information that readers could assess in relation to their own contexts rather than assigning relevance scores directly. Directions for future research appeared in 40 guides, guidance for educational practice in 27 guides, and theories, conceptual frameworks, or educational principles in 11 guides. The author teams included clinicians in 37 guides, librarians in 16 guides, and trainees in 11 guides. Consultation with external experts informed 25 guides. Official updates were issued for 3 guides at 10, 4, and 9 years after initial publication. Because STORIES is a reporting standard and the analysis focused on BEME Guides, the results are best interpreted in terms of the information provided by the guides examined. The availability of open data and the extent of its actual use by educators are separate measures. The study recommended consistent open access, greater stakeholder participation, and the inclusion of educational theory.

Introduction and Continuous Adjustment of Clinical Reasoning Education

A longitudinal study of a clinical reasoning education intervention detailed how evidence was introduced and applied within a curriculum (Chamberland et al., 2022). The study was conducted within the new 4-year CBME curriculum at the University of Sherbrooke in Canada, taking into account 206 students, three geographically separated educational sites, and limited faculty resources. The research team used the Knowledge to Action (KtA) framework, which encompasses knowledge creation and an action cycle for application. The application process included seven steps: identifying the gap between research and practice and selecting evidence, adapting it to the local context, assessing barriers and facilitators, implementing the intervention, monitoring use, evaluating outcomes, and sustaining use. Clinical reasoning was viewed as a process of organizing knowledge around illness scripts. The team selected self-explanation (SE), in which learners verbally explain learning content, and structured reflection (SR), in which they compare and contrast possible diagnoses. SE-SR comprised 11 sessions of 90-minute web-based activities completed individually during integration weeks over the first 2.5 years of the curriculum, with each session covering three challenging clinical cases.

Cases written by clinical faculty were reviewed by educators and curriculum planners. Following preliminary checks of materials, platforms, and procedures with student volunteers from the previous academic year, the cases were introduced as a mandatory activity in October 2017. The study used a mixed-methods concurrent triangulation design, analyzing quantitative and qualitative data separately and then integrating them at the interpretation stage. A total of 175 students, representing 85% of all students, agreed to anonymous data collection on the platform. During the study period, 2 dropped out, 5 repeated an academic year, and 11 took a leave of absence. Student focus groups involved 5 students in the middle of year 1, 24 at the end of year 1, 15 in the middle of year 2, and 11 at the end of year 2. There were also 15 stakeholder participants. A research assistant not involved in the program conducted the interviews. The research team coded the data and reviewed the themes. Findings were integrated according to six implementation outcomes from Proctor et al.: fidelity, feasibility, appropriateness, acceptability, adoption, and penetration. Student surveys and interviews, along with stakeholder feedback, led to additional time for cases, adjustments to feedback timing, a longer period of platform access, and changes to the recording method.

Excluding cases 4–6, the average completion rate was 81% for cases 1–21 and 92% for cases 22–33 following the change in recording method, with an average time spent per case of 24:08 minutes. Because some SE data were initially lost to recording errors, the change in completion rates is best interpreted alongside the improvement in the recording method. Completion rates of 25–53% for cases 4–6 reflected the option to complete only one of the three cases to ease the learning burden. Students described the activity as helpful for assessing their knowledge, identifying learning priorities, and preparing for exams and clinical work. They reported becoming familiar with it through repetition. Stakeholders considered the activity aligned with curriculum goals and manageable through monitoring and adjustment once the platform was established. The quality of SE recordings and SR texts was directly evaluated in a separate study.

Reading notes

Whether a theory is reported in a guide is distinct from whether it was used in the primary studies. Time spent was an indirect indicator of fidelity. The student interviews captured the experiences of a small number of volunteers. It was difficult to isolate and establish the contribution of KtA itself to implementation success. From the perspective of organizational learning in HRD, examining how students’ and staff members’ experiences inform time allocations, feedback, and platform operating procedures offers a focus for research.

References

• Chamberland, M., Setrakian, J., Bergeron, L., Varpio, L., St-Onge, C., & Thomas, A. (2022). Harnessing a knowledge translation framework to implement an undergraduate medical education intervention: A longitudinal study. Perspectives on Medical Education, 11(6), 333–340. https://doi.org/10.1007/s40037-022-00735-7
• Maggio, L. A., Thomas, A., Chen, H. C., Ioannidis, J. P. A., Kanter, S. L., Norton, C., Tannery, N. H., & Artino, A. R., Jr. (2018). Examining the readiness of best evidence in medical education guides for integration into educational practice: A meta-synthesis. Perspectives on Medical Education, 7(5), 292–301. https://doi.org/10.1007/s40037-018-0450-9