Generative AI in medical education: Uses and open questions

Generative AI responses and learner changes

Generative AI is used in medical education to answer questions, create learning materials, and provide feedback on performance or conversations with virtual patients. Understanding the educational use of generative AI requires distinguishing among the quality of model-generated responses, how students use the tools, and subsequent changes in performance. Key considerations include the accuracy of generated information, what students practice again after receiving feedback, and how they perform when interacting with actual patients.

From test-taking abilities to educational possibilities

A scoping review synthesized early discussions of generative AI use in medical education using Arksey and O’Malley’s framework (Preiksaitis & Rose, 2023). The research team searched PubMed, Web of Science, and Google Scholar for English-language literature published between January 1, 2022, and June 21, 2023. Publications that addressed AI capable of generating new content were selected. Literature dealing exclusively with education for non-physician professions or non-generative AI was excluded. After removing duplicates and materials that did not meet the selection criteria from the 2761 retrieved records, the team analyzed 41 papers. All were published in 2023. Two researchers independently screened the titles and abstracts of the first 100 papers, yielding a Cohen kappa of 0.76. The team analyzed the literature thematically using Braun and Clarke’s methodology. Among the 41 papers analyzed, 21 (51.2%) were opinion pieces and 20 (48.8%) were original research articles, with 16 of the original research articles measuring generative AI performance on standardized assessments.

The examination studies covered the United States Medical Licensing Examination, specialty exams in cardiology, neurology, and ophthalmology, and undergraduate subjects such as parasitology and biochemistry. Performance was generally high, although the model did not pass some examinations, including the Taiwanese family medicine board examination. Proposed educational applications included personalized learning plans, material generation, feedback, and simulated conversations to practice breaking bad news about cancer diagnoses. Two papers on image generation addressed case-based learning in plastic surgery and reflective activities in medical humanities. They discussed the potential to reduce concerns about privacy in clinical photographs and copyright protection for existing artworks. Writing, translation, and literature summarization were presented as functions that could support non-English-speaking researchers. The literature discussed hallucinations that produce nonexistent information and references, limited access to up-to-date information, bias, academic integrity, and differences in accessibility. It examined the possibility that delegating tasks to AI could reduce opportunities to practice critical thinking or problem-solving.

The included literature contained no empirical studies explicitly examining learning processes or evaluating educational effects when learners used generative AI. AI literacy, assessment, and human-AI interaction were proposed as areas for further research. AI literacy encompassed tool use, the evaluation of evidence and bias, responsible use of information, and the ability to judge its potential effects on clinical behavior. The “black box” issue, in which the process of generating responses is difficult to observe, was linked to teaching learners how to judge uncertain clinical information. For formative assessment, the authors proposed structuring learning activities around AI explanations and learners’ subsequent questions. For assessments using AI, they proposed pilot testing, collecting validity evidence, and examining how assessment results affect individuals. The reported examination scores in early English-language literature are data on models’ response capabilities. Personalized learning and feedback remain educational proposals requiring further evaluation. The Context, Input, Process, Product (CIPP) model, theory-driven evaluation, and realist evaluation were presented as methods for evaluating these proposals in relation to learners’ attitudes, experiences, and educational contexts.

Students’ generative AI use through six learning types

A systematic review of students’ use of generative AI applied Laurillard’s Conversational Framework to classify that use into acquisition, inquiry, practice, production, discussion, and collaboration (Pham et al., 2025). The research team conducted the review in accordance with PRISMA 2020 and registered the protocol in PROSPERO (CRD42024543726). The team searched ERIC, Education Database, Ovid Medline, Ovid Embase, and Scopus through September 15, 2024, with no restriction on the start year. Peer-reviewed original research on student use in formal university programs in health professions education was selected. Of the 1924 papers retrieved, 33 were included, with 28 published in 2024. These comprised 13 papers in medicine, 9 in nursing, 4 each in dentistry and pharmacy, 1 in veterinary medicine, and 1 each addressing multiple fields or medicine and pharmacy combined. ChatGPT was the predominant tool. The mean MERSQI score was 10.4 with a range of 1–18. No papers were excluded based on quality assessment results. Five members of the research team independently classified the use cases into the six learning types using deductive content analysis, reaching consensus after a meeting and review involving all authors.

Practice was reported in 24 papers (about 73%), inquiry in 23 (about 70%), production in 22 (about 67%), and acquisition in 18 (about 55%), while discussion and collaboration were each reported in 4 papers (about 12%). The denominator for these percentages is the number of included papers, not the number of students or learning hours. Acquisition activities included explanations of concepts, correction of misconceptions, summaries of lecture notes, and translations. Inquiry activities included gathering and organizing information, exploring alternatives, searching the literature, and generating project ideas. Of the 24 papers addressing practice, 17 included interactions with virtual patients, history taking, communication, and practice in interprofessional collaboration. Structured prompts effectively simulated patient situations, whereas open-ended or ambiguous questions tended to elicit responses that were less clear and less plausible. Among the 22 papers covering production, 20 used generative AI to complete assignments. Some studies reported improvements in grades and assignment completion times. Students also used tools to refine the structure and grammar of essays and case reports. They sometimes judged the tools to be better suited to grammar correction and translation than to complex academic writing.

Discussion included conversations with AI. Collaboration was restricted to activities in which students received AI support while completing joint tasks with other people. Some discussion activities allowed students to express opinions with less concern about others’ evaluations. Findings on collaboration included both positive outcomes for teamwork and results showing no significant effects. The authors interpreted the frequent reporting of individual activities as a pattern of use that supported personalized, self-directed learning. The number of papers addressing discussion and collaboration does not directly indicate whether students’ peer interactions decreased. The review considered potential publication bias, the restriction to English-language literature, and the possible influence of negative perceptions of AI users on self-reporting. The authors proposed education in information verification and prompt engineering, institutional policies for ethical use, and experimental and longitudinal studies.

Combining automated assessment and peer feedback in national licensure preparation

A prospective deployment study examined the use of CPX-MATE (Clinical Performance eXamination with Medical Students’ Assistant for Training and Evaluation), which was made available for independent use during exam preparation at Yonsei University College of Medicine (Song et al., 2026). This platform combines a real-time evaluator (RTE) that scores peer role-plays with a virtual standardized patient (VSP) for solo conversation practice. Researchers set the schedules in preliminary validations. The research team examined adoption and sustained use when the platform was offered free of charge without instructor scheduling or guidance for 60 days from February 22 to April 22, 2026. Of the 72 eligible participants, 43 (59.7%) consented. The students had completed approximately 40 formative OSCE practice sessions and three summative mock exams during their clinical years. The content comprised 55 chief complaints and 110 VSP scenarios. The checklists were reviewed by 2 faculty members, and the AI-generated scenarios by 3 physicians. The RTE used Whisper-1 for transcription and GPT-5.4 for item-by-item dichotomous scoring. The evaluation followed Proctor et al.’s implementation outcomes framework. It drew on logs, pre- and post-use surveys, API costs, and interviews with 14 individuals. Acceptability and appropriateness were measured using the 5-point AIM and IAM.

Forty-two students used the platform at least once, yielding an adoption rate of 97.7% (42/43) among those who consented and 58.3% (42/72) among all eligible participants. Of the total 3,042 sessions, the RTE accounted for 2,807 and the VSP for 235. The overall completion rate was 97.9%. The cumulative usage time of 745.1 hours included 178.6 hours spent rechecking results in Score Review. The reuse rate was 100% among RTE users and 72.0% among VSP users. Median RTE acceptability increased from 4 before use to 4.62 after use (Hodges–Lehmann [HL] estimate 0.38, 95% CI 0.25–0.50; P = 0.003). The correlation between prior attitudes and usage intensity was not significant, although survey responses clustered at high scores. Across 776 student–chief complaint combinations practiced two or more times, the HL estimate for the change in total score between the first and second performances was +1.67 points (95% CI 0.99–2.23; P < 0.001; r = 0.191). The increase occurred primarily in history taking. The correlation between time spent in Score Review and changes in history taking scores was ρ = 0.134. In 40 independently re-evaluated sessions, the overall Gwet’s AC1 for agreement between expert and automated scoring was 0.959. Total API operating costs were $592.45, excluding development, maintenance, faculty review, scenario writing, and institutional adoption.

All 14 interview participants had strategies for addressing their weaknesses through practice before using the platform. They described peer and RTE feedback as complementary. Peers observed affect and clinical flow, while the RTE checked for checklist omissions and the accuracy of content recall. Coordinating feedback became less burdensome for 12 participants. Seven students used the records to improve their practice. The process of identifying repeated omissions across sessions to guide further practice was directly confirmed in 2 individuals. In a setting where students role-played in groups of four for more than 6 hours a day, the RTE became part of existing practice. The VSP was used when peers were unavailable or to check memorized content verbally. Of the 12 participants who changed their practice focus immediately before the exam, 10 mentioned time management and none identified history taking. Because the study observed voluntary use without a control group, score increases are best interpreted in light of general practice and retesting, familiarity, and reduced test anxiety. The study had a single-institution setting and used purposive sampling for interviews. The RTE checklist did not directly assess clinical reasoning. Usage records and interviews describe how automated feedback becomes part of existing deliberate practice.

Reading notes

From an HRD perspective on self-regulated learning, AI usage volume or time spent viewing feedback can be supplemented with data on how students set goals, identify weaknesses, and adjust subsequent practice to examine how they manage their learning. The process of turning omissions identified in AI feedback into subsequent practice goals, and applying that practice to actual history taking and communication, offers a specific focus for studying self-regulated learning and learning transfer together. Instances in which peers observe affect and clinical flow while AI checks for item-by-item omissions provide data for examining educational designs that combine human and AI feedback.

References

• Preiksaitis, C., & Rose, C. (2023). Opportunities, challenges, and future directions of generative artificial intelligence in medical education: Scoping review. JMIR Medical Education, 9, e48785. https://doi.org/10.2196/48785
• Pham, T. D., Karunaratne, N., Exintaris, B., Liu, D., Lay, T., Yuriev, E., & Lim, A. (2025). The impact of generative AI on health professional education: A systematic review in the context of student learning. Medical Education, 59(12), 1280–1289. https://doi.org/10.1111/medu.15746
• Song, J. W., Kim, M., Lee, Y. G., Cheong, S. W., Kim, J. J., Cho, J., Kim, Y. S., You, S. C., & Park, C. (2026). Naturalistic adoption and deliberate practice use of an AI-based OSCE platform during national licensure preparation. npj Digital Medicine. article in press (published online 2026-07-15; volume/article number not yet assigned in Crossref). https://doi.org/10.1038/s41746-026-03024-3