Powered by AI models trained on troves of text pulled from the internet, chatbots such as ChatGPT and Google’s Bard responded to the researchers’ questions with a range of misconceptions and falsehoods about Black patients, sometimes including fabricated, race-based equations, according to the study published Friday in the academic journal Digital Medicine.
Experts worry these systems could cause real-world harms and amplify forms of medical racism that have persisted for generations as more physicians use chatbots for help with daily tasks such as emailing patients or appealing to health insurers.
The report found that all four models tested — ChatGPT and the more advanced GPT-4, both from OpenAI; Google’s Bard, and Anthropic’s Claude — failed when asked to respond to medical questions about kidney function, lung capacity and skin thickness. In some cases, they appeared to reinforce long-held false beliefs about biological differences between Black and white people that experts have spent years trying to eradicate from medical institutions.
While I think you can get ChatGPT to be racist if you work at it, I’ve been toying with it in terms of the validity of its medical information and it’s pretty damn good.
I can ask extremely complex questions and then follow those up with clarifying questions and it’s consistently been accurate. I’m by no means using it to practice medicine and for the love of god nobody should be going to it for medical advice.
But it’s obvious this is going to be used in hospitals within the next 5 years. They just have to sanitize the information being put out and ensure it can only pull from reliable sources.
It’s really crazy how good it is at explaining complex topics. It’s helped me understand medical concepts that were confusing to me (I’d ask it to explain something better so I can understand it and then I’d verify what it was saying with UpToDate, a database of accurate medical information). Even just having it reword something or having it spit out ways to remember a concept worked really well.
Again, it’s not there yet and cannot be trusted, but it will be able to be trusted soon enough and it’s going to change medicine.
AI medical chatbots will soon be bundled in EMRs and doctors are going to be able to punch in questions on the fly to get accurate and specific answers to the issues that arise.
