IBIIS-AIMI Seminar: Healthcare AI Requires System Changes, Not Just Tasks - Susan Shelmerdine, MBBS, PhD

Abstract: Most healthcare AI research asks a narrow question: can the algorithm perform the task well? Increasingly, the answer is yes. But task performance is insufficient when that algorithm enters a living clinical system - one shaped by human cognition, professional identity, institutional culture, and trust. This talk shares a case study within the NHS of how an AI entering a system succeeded and also failed (depending on your definition) - not because of the technology, but the people and structures it encounters. Healthcare AI's real risks are not technical - they are increasingly behavioural. Automation bias, deskilling, and the quiet erosion of clinical reasoning do not announce themselves; they accumulate. And we are not routinely detecting them. Much of the work is early and speculative, but I make the case for reflecting on human factors evaluation for AI deployment.

About: Susan Shelmerdine is a consultant pediatric radiologist at Great Ormond Street Hospital for Children, Associate Professor at UCL, and Hon Professor at City St George’s University of London. She is an NIHR funded Advanced Fellow and her research work has included AI model development, evaluation and testing for children’s fracture detection on X-rays; but also real-world deployment, governance and post-market monitoring of AI tools for adult CXR imaging. She founded and chaired the European Society of Pediatric Radiology AI Taskforce (2020–2025), sits on the Royal College of Radiologists AI Advisory Group, is a member of the expert review panel for the MHRA (UK regulatory body), and has recently launched an AI Youth Council seeking to involve youth voices in how AI is used in healthcare. She has published over 167 peer-reviewed papers including work in The Lancet, NEJM AI and BMJ, and was the recipient of several awards including the RCR Roentgen Professorship (2025), AI100 UK, DHSC AI Visionary Award, and RCR Exceptional Contribution Award.

Attendance is open to the Stanford and AIMI affiliate community. Please contact aimicenter@stanford.edu for the Zoom link if you would like to attend virtually.