Beware of Dr. Chatbot: Generative AI Often Gives Unreliable, Biased Medical Advice
New study shows patients shouldn’t put their full trust in the current state of general purpose artificial intelligence
SAN FRANCISCO — A group of ophthalmologists assessed the utility of the most popular generative artificial intelligence (AI) programs – ChatGPT, Google Bard, and Bing Chat – as an educational resource for patients with eye conditions and disesases. They found that a majority of responses from all three tools were inaccurate, and that two of the three chatbots demonstrated a significant bias against female ophthalmologists. The authors caution that these models are not fine-tuned to provide accurate eye health information. They fear the inaccuracies may further delay treatment, putting their patients at increased risk for vision loss. The study will be presented this week at AAO 2023, the 127th annual meeting of the American Academy of Ophthalmology.
Researchers from the University of Southern California asked three practicing ophthalmologists to compare the ability of these programs to answer common patient questions and create educational resources, as well as recommend ophthalmologists practicing in the 20 largest cities in the United States. The ophthalmologist evaluated the information on a scale of 1 to 4.
Google Bard scored the highest for quality and accuracy of responses to patient questions, with an average rating of 2.3 out of 4. ChatGPT had the highest rating for patient educational resources, 3 out of 4.
All three chatbots struggled when asked to recommend practicing ophthalmologists or to accurately locate ophthalmologists in or near a specific city. Google Bard and Bing Chat recommended female ophthalmologists less than 2 percent of the time, even though 27 percent of the nation’s ophthalmologists are women.
“Given the substantial bias and inaccuracy demonstrated in this study, we warn against reliance on AI chatbots when seeking health-related information until improvements in algorithms are achieved and validated in the future,” said researcher Michael Oca, BS of the University of California, San Diego. “A poor recommendation from a chatbot could further delay a patient’s treatment.”
For accurate, ophthalmologist-vetted information online, the Academy offers www.EyeSmart.org as a public resource.
“Relying on online tools for quick advice may be tempting, but we urge the public to remember that this is not a replacement for a comprehensive eye exam with an ophthalmologist. Seeing a medical doctor for preventative exams and examining any sudden change in vision is the best way to protect your eye health,” said senior author Sandy Zhang-Nunes, MD, associate professor of clinical ophthalmology, and director of oculofacial plastic surgery at the University of Southern California.
About the American Academy of Ophthalmology
The American Academy of Ophthalmology is the world’s largest association of eye physicians and surgeons. A global community of 32,000 medical doctors, we protect sight and empower lives by setting the standards for ophthalmic education and advocating for our patients and the public. We innovate and support research to advance our profession and to ensure the delivery of the highest-quality eye care. Our EyeSmart® program provides the public with the most trusted information about eye health. For more information, visit aao.org.