In 2023, artificial intelligence has been a huge story.
This was the year large language models like OpenAI’s ChatGPT program were unleashed on the masses, giving people access to AI programs that can help with everything from figuring out what to eat for dinner tonight to explaining complex topics theories.
Related: OpenAI CEO Sam Altman says that ChatGPT is not the way to superintelligence
But ChatGPT, and every other large language model (LLM), is only as good as the information which feeds it. And a new study suggests that artificial intelligence still has much to learn about the medical field.
Over a 16-month period between 2022 and 2023, Sara Grossman, PharmD, Associate Professor of Pharmacy Practice at Long Island University, and her team challenged ChatGPT with real medical questions posed to Long Island University’s College of Pharmacy drug information service.
The study’s conclusion?
“Healthcare professionals and patients should be cautious about using ChatGPT as an authoritative source for medication-related information,” Grossman. “Anyone who uses ChatGPT for medication-related information should verify the information using trusted sources.”
Pharmacists involved in the study first researched and answered 45 queries and each answer was reviewed by a second investigator. The investigators then asked ChatGPT the same questions — minus six questions because “there was a lack of literature to provide a data-driven response.”
Of the 39 questions that the team asked the free version of ChatGPT, only 10 of the responses were judged to be satisfactory.
In one instance, ChatGPT was asked whether a drug interaction exists between Covid-19 treatment Paxlovid and blood pressure lowering medication verapamil. Chat GPT said that there were no interactions that had been reported for that combination of drugs.
“In reality, these medications have the potential to interact with one another, and combined use may result in excessive lowering of blood pressure,” Grossman said. “Without knowledge of this interaction, a patient may suffer from an unwanted and preventable side effect.”
For the 29 questions ChatGPT missed, researchers determined that the program did not directly answer 11 of them while 10 were inaccurate and/or 12 were incomplete.
Unlock the secrets of the stock market with Real Money Pro. Gain direct access to elite portfolio managers’ top stock picks and time-tested investment strategies. Embark on your journey to financial success — start now.
