Conference Report: AMEE goes big on AI in Barcelona
My reflection on my first AMEE conference, where AI research is everywhere, but widespread adoption still lags
This was my first time attending AMEE, held this year in Barcelona, with around 4,000–5,000 delegates. The theme, How are educators relevant to health?, felt particularly timely given the prominence of AI throughout the programme.
AI featured heavily in the academic sessions, with bold and often inspiring research on display. By contrast, there was little presence among the commercial exhibitors. This gap suggests that while enthusiasm for AI in education is high, widespread adoption still lags behind. For now, I think educators remain the key agents of translation, deciding which technologies truly serve learners and patients, and how. We’ll see how long that lasts…
I’m looking forward to learning how AI adoption matures next year at AMEE 2026 in Vienna.
I was pleased to contribute to a session on AI and Digital Assessment, sharing work co-authored with Sandhya Duggal (who contributed a guest article last week) and Jane Illes. Our paper, Evaluating the Reliability and Consistency of a Custom GPT-4-Based Marker of Student Assignments, tested whether a large language model could provide stable scoring of long-form written work. Using published critical reviews as proxies for dissertations, we asked the AI system to mark each one 100 times. The results showed strong internal reliability (Cronbach’s alpha >0.8), no evidence of bias or drift across repeated attempts, and levels of variation similar to those seen with human markers. While we have not yet tested accuracy against a human “gold standard”, the findings suggest AI could act as a reliable second marker or sense-checker. I think the application for this work is in supporting, rather than replacing, the educator.
It was fascinating to see related work from colleagues at King’s College London, where Helen Oram and her team are developing their own model for marking student assignments. That parallel effort highlights how many institutions are now taking ownership of AI in assessment, exploring how these tools can be adapted to their contexts rather than simply adopted off-the-shelf.
☀️ Outside the conference, there was time to explore Barcelona: wandering the Gothic Quarter, taking in the architecture, and enjoying ridiculous amounts of tapas. While very impressive, the Sagrada Família is still under a little scaffolding 140 years after construction began. Must be waiting on Reviewer 2’s comments.
