AI-Generated Medical Imagery and Representation: Do We All Get to be Seen?
A student’s take on our study testing whether AI image generators fairly represent skin tones in dermatology.
🎓 It is my pleasure to welcome another guest contributor to AI × MedEd: Ayla Ahmed graduated in the summer from University of St Andrews School of Medicine, and worked with us on a project investigating biases in generative AI outputs.
Generative AI can turn a line of text into a lifelike image in seconds. In medical education that’s exciting; we can create tailored cases for anatomy and dermatology without expensive equipment or consent for imaging. But representation matters. Dermatology has long underrepresented darker skin in teaching images, and that gap shows up in care. Because many generative AI models are trained on existing images and captions, they may mirror the same skew we see in textbooks, where darker skin is underrepresented. There are rising concerns that rather than acting as a resourceful replacement, images generated by AI will simply carry these biases forward perpetuating the loop of inequitable patient care.
In 2024 our team asked two popular generators (DALL·E 3 and Midjourney) to create 100 psoriasis images each. We also built a custom GPT that injected real US demographic proportions into the DALL·E 3 prompt to see if that reduced bias. We scored the skin tone visible in each image using the 10-point New Immigrant Survey scale.
Our study found that “off-the-shelf” AI models did not produce a range of skin tones reflective of the general population of the United States. Of the 200 images produced by DALL·E 3 and Midjourney, only one depicted dark skin.
Chi-square tests showed that the distribution of skin tones in AI images from the “off-the-shelf” models differed significantly from the 2012 ANES benchmark (i.e. the US Population). This difference was driven by under-representation of darker skin tones. This shows not only the lack of diversity demonstrated by the AI models, but also their disproportionate preference for lighter skin tones.
However, the injection of real-world demographic data can correct this inherent bias. In contrast, the distribution of skin tones in images produced by the Custom GPT, equipped with a more sophisticated prompt and data, showed no statistically significant difference compared to the ANES data. This demonstrates that prompt injection of real demographic data can override the lack of representation resulting from the underlying training data.
When we think about the efforts taken to address the issues of representation in medicine, it is important that we are aware of what actions will truly be able to counteract the deep systemic biases that exist. For skin tone representation actions such as review committees that ensure adequate representation and increasing efforts in skin tone range in publications are unlikely to overcome the vast training material already in circulation. However, incorporation of real-world demographic data into post-training prompts appears to be effective.
It is also worth tailoring prompts to the population you teach. By adjusting demographic instructions to reflect the composition of your region or service, AI can generate images that better match local diversity. That brings two benefits: it addresses representation and aligns cases with the real-world contexts your students and practitioners will encounter.
These results emphasise the importance of conscious and deliberate efforts in AI development to ensure diversity and prevent the reinforcement of existing biases. New technologies are advancing at an unprecedented pace, unlocking possibilities for transformative medical breakthroughs. Yet progress is not truly progress if it only serves a select demographic. The persistent lack of representation, such as the underrepresentation of darker skin tones in image generation, highlights a critical gap. If left unaddressed, these innovations risk reinforcing inequities instead of dismantling them. For technology to fulfil its promise, it must be built to benefit everyone, not just those it most easily recognises.
If you enjoyed this write-up, you might also enjoy this roundup about AI-generated medical imagery:
 |
|