Asian senior or elderly old lady woman patient use toilet bathroom handle security in nursing hospital ward
Urology is an important concern for seniors, but ChatGPT might not be the best go-to source for answers. (photo credit: sasirin pamai/Getty Images)

Although new studies continue to tout artificial intelligence’s successes at interpreting medical data and offering advice, it appears to be lacking in at least one medical category important for older adults: urology. 

When asked common medical questions patients might query a physician about urinary complications, the popular chatbot ChatGPT fell far short of clinical standards, and in many cases, the information it provided was not accurate at all. 

This is the first time ChatGPT has been evaluated specifically for urology, the researchers claim. 

But the potential for AI to offer inaccurate, or biased, healthcare information has been a major concern ever since use of the tech exploded earlier this year. 

Perhaps because of this, patients also remain concerned about AI’s clinical uses

Some of the most common conditions older adults have that lead to urinary complications include an enlarged prostate (symptoms are present in about half of men age 75 and older) and dementia. Urinary tract infections themselves can also lead to further complications in seniors such as delirium or abnormal heart rate. 

In the urology study, which was conducted at the University of Florida, Gainesville’s Urology Department, not only did ChatGPT often provide inaccurate information, such as incorrect treatment recommendations, but it did so with “confidence,” the researchers stated

Other results were equally damning: The chatbot offered responses the researchers considered “appropriate” only 60% of the time. 

“ChatGPT misinterprets clinical care guidelines, dismisses important contextual information, conceals its sources, and provides inappropriate references,” the researchers concluded