I wonder: Could AI have shortened my son’s 8-year long diagnostic odyssey?
By Helene M. Epstein
I was intrigued by an article in JAMA Internal Medicine which determined that ChatGPT’s responses to patient inquiries were better quality and more empathetic than human physicians’ responses, over three-quarters of the time. The evaluators were physicians, and the 195 queries were sourced from a subreddit, known as r/AskDocs. I checked the study’and the responses were evaluated blind, meaning without any prior knowledge of which ones came from the experimental AI chatbox and which were written by actual physicians answering the same questions.
This is the opposite of what we have come to expect from the present state of artificial intelligence. Up to now, employment of AI resources was mostly limited to shortening the differential of medical conditions, which means the list of potential diseases or conditions a physician should consider before running diagnostic tests. Longtime existing artificial intelligence and digital clinical decision support companies like visualDx and Isabel are careful to say they are a diagnostic tool, not providers of medical advice to patients. The de facto modifier on every AI medical tool states that their information is best evaluated by a trained healthcare professional. ChatGPT also dances around the issue and won’t directly diagnose anyone.
Artificial Intelligence has proven unreliable or inconsistent for use as a diagnostic Magic 8-Ball. We know that even IBM wasn’t able to create a reliable AI diagnostic tool. Just look up IBM’s Watson for pages on why that AI attempt failed.
"I wondered if ChatGPT might have helped when we were still struggling to identify the cause of my son’s progressive and bizarre symptoms."
There have been many papers and investigations that show AI is just as biased as human physicians, probably because it is programmed by humans with biases, and because the current standard of care resources in the United States include too many implicit biases. The types of bias we commonly see in diagnostic reasoning, include, but are not limited to, racial misconceptions, lack of understanding about women’s bodies, dismissal of symptoms by the elderly as just a sign of aging, unwillingness to engage deeply in different cultural needs of minority and immigrant populations, and much more.
Personally, I haven’t had high hopes for medical AI solutions this decade. If the AI on my mobile phone, you know her as Siri, is incapable of understanding many basic commands, how can an AI possibly diagnose human beings whose symptoms rarely present in a standardized way?
On the other hand, since no one human can recall all the myriad presentations of disease, perhaps a highly functional live learning machine can at least get us in the right ball park. As Mark Graber MD has often said, there are over 10 thousand diseases but only a few hundred symptoms. And medical schools routinely teach only a thousand of these diseases.
So, I wondered if ChatGPT might have helped in 2008 when we were still struggling to identify the cause of my son’s progressive and bizarre symptoms. We had tried and failed for over eight long years, working with multiple pediatric specialists, before we were able to get my son diagnosed and treated for the mystery illness that was misdiagnosed again and again and again by physicians dismissive of our concerns and overly certain of their approaches.
Here's an excerpt of my “conversation” with the AI experiment. I’ll note that over the eight years of my son’s diagnostic journey, we did everything the AI suggested but the specialists did not. For example, the only objective testing done by the three pediatric neurologists we consulted was two spread out sleep tests that were inconclusive and unhelpful.
HME
What type of medical expert should I speak to about my son's progressive symptoms? He started out with some compulsive behavior, ADHD, then Tourette's type vocal symptoms, joint pain, now he's suddenly stopped growing, has memory loss, and keeps falling asleep in class.
ChatGPT It sounds like your son's symptoms are quite complex and varied, and it would be best to seek out the help of a medical professional who can provide a comprehensive evaluation and diagnosis.
Given the range of symptoms you've described, it would be a good idea to consult with a pediatric neurologist. They specialize in disorders that affect the nervous system, including the brain, spinal cord, and nerves.
A pediatric neurologist will be able to perform a thorough evaluation of your son's symptoms, which may include a physical exam, blood tests, imaging studies (such as an MRI), and other diagnostic tests as needed. They will also be able to work with you to develop a treatment plan that addresses your son's specific needs and concerns.
In addition to consulting with a pediatric neurologist, you may also want to consider seeking out the help of other specialists, such as a pediatric rheumatologist (for the joint pain) and a pediatric endocrinologist (for the growth issues and memory loss). Your son's primary care physician can help coordinate these referrals and ensure that he receives the appropriate care.
I decided to probe with further questions, something we could not have done with my son’s medical providers. They just never had enough time to give us.
HME
What would you do?
ChatGPT As an AI language model, I am not able to provide specific medical advice or recommendations. However, I can provide some general suggestions:
Talk to your doctor: If you have concerns about your son's diagnosis or treatment plan, it's important to talk to your doctor about your questions or concerns. You can ask for more information about how they arrived at the diagnosis, what the treatment plan involves, and any potential side effects or risks.
Seek a second opinion: If you feel that your doctor is not addressing your concerns or if you would like a second opinion, consider seeking out another medical professional. You can ask for a referral from your current doctor or search for specialists in your area who have experience with your son's symptoms.
Educate yourself: Learn as much as you can about your son's condition, including symptoms, potential causes, and treatment options. This can help you better understand the diagnosis and treatment plan, as well as give you the tools to advocate for your son's care.
Keep track of symptoms: Keep a log of your son's symptoms, including when they occur and how severe they are. This can help you and your doctor track any changes in symptoms over time and adjust the treatment plan as needed.
Remember, it's important to work closely with your son's medical team to ensure that he receives the best possible care. By advocating for your son's health and staying informed, you can help him get the care he needs to feel better.
Now I’m impressed. This is exactly the advice experts would give. And it’s the kind of advice I would give as well. However, it wasn’t enough when my son was ill so I probed further.
HME
I've done all of what you've recommended and it hasn't worked. We have been trying to find an answer for eight years! We've seen three pediatric neurologists, two pediatricians, a rheumatologist, an endocrinologist, and an infectious disease specialist. Are there any additional thoughts you might have on how we can arrive at an answer?
ChatGPT I'm sorry to hear that you and your son have been struggling to find an answer for so long despite seeing several medical professionals. It can be frustrating and disheartening when medical conditions are difficult to diagnose and treat.
Here are a few additional thoughts and suggestions:
Consider seeking out a medical center or hospital with a multidisciplinary team of specialists who can work together to evaluate and treat complex cases like your son's.
Request a referral to a research hospital that has a focus on the particular medical condition(s) that your son is experiencing.
Contact advocacy groups or foundations that focus on your son's specific symptoms or conditions, as they may be able to provide support, resources, and information on other medical professionals or research institutions to consider.
Explore genetic testing to see if there is a genetic component to your son's symptoms.
Keep detailed records of your son's medical history, including test results, treatments tried, and symptoms experienced. This can help you and future medical professionals see a complete picture of your son's medical journey and may provide important clues for diagnosis and treatment.
Remember that finding a diagnosis for complex medical conditions can take time and persistence. It's important to continue advocating for your son's health and seeking out medical professionals who are willing to collaborate with you to find answers.
This was all excellent advice but we’re no closer to an answer, so I asked a more specific question.
HME
So, what do you think might be the top five diagnoses we should explore? [NOTE: If we were in an appointment with a real doctor, I would try to help broaden his thinking by asking, “What else do you think this might be?”]
ChatGPT As an AI language model, I cannot provide a specific diagnosis or predict what medical condition your son may have. However, based on the symptoms you've described, some potential conditions that may be considered and investigated by a qualified medical professional include:
Autoimmune disorders: Given the joint pain, sudden growth cessation, and other symptoms your son has been experiencing, an autoimmune disorder could be a possibility. Conditions such as systemic lupus erythematosus (SLE), rheumatoid arthritis (RA), and juvenile idiopathic arthritis (JIA) can cause a range of symptoms that may overlap with your son's symptoms.
Neurological disorders: Your son's history of ADHD, Tourette's-like vocal symptoms, memory loss, and falling asleep in class suggest that a neurological disorder may be a possibility. Conditions such as epilepsy, narcolepsy, or encephalitis may need to be considered.
Infectious diseases: Although your son has not had a positive strep test, there may be other infectious diseases to rule out, particularly if he has other signs of infection such as fever, headaches or a skin rash. Some examples include Lyme disease or tick-borne infections, chronic viral infections or other bacterial infections.
Endocrine disorders: The sudden cessation of growth could suggest a problem with your son's endocrine system, which is responsible for regulating hormones in the body. Conditions such as growth hormone deficiency, hypothyroidism, or hyperthyroidism could be investigated.
Mitochondrial disorders: Mitochondrial disorders can cause a wide range of symptoms, including muscle weakness, seizures, developmental delays, and other neurological symptoms.
Again, it's important to work closely with a qualified medical professional to evaluate your son's symptoms, medical history, and any relevant test results to arrive at a correct diagnosis.
HME
Thank you very much.
ChatGPT You're welcome! I'm here to help if you have any further questions or concerns. Best of luck to you and your son in your medical journey.
My conclusions are simple. There are two reasons why the AI provided me with better quality answers than any of the clinicians who failed repeatedly for my son. First, the AI was able to access and synthesize a tremendous amount of data and then organize it and communicate it to me in a reasoned and comprehensible manner. Second, it was able to accomplish all that and still be empathetic and project caring in just a few seconds.
In the US system of healthcare, no clinician is permitted the time to accomplish the same. We are in a highly corporatized and profit-based system based on insurance and CMS reimbursements. Communicating at this level would prevent them from seeing the next patient and covering their expenses. Also, few physicians seem to have the same range of knowledge at their fingertips. I know some brilliant primary care physicians with ready access to troves of data in their own memory banks but sadly, they are the exception. Primary care physicians should be able to at least direct the patient to the right specialists. I know many who would and do, though my son’s pediatrician only sent us to specialists when I asked specifically for the referral based on my own hours of reading clinical studies and studying medical texts. In the many emails and comments I receive from patients, it seems referrals are often limited to the healthcare corporation the primary care doctor works with. Their associates might not be the best specialists to recommend.
ChatGPT was unable to diagnose my son but had it existed when he was a teen, it might have at least helped his physicians take his symptoms more seriously. The AI didn’t dismiss us. My son’s doctors did and he suffered for it.
