I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor

It doesn't help when doctors are often unaware of outliers affecting the test results. E.g. I've had a number of doctors freak out over my eGFR (kidney function) test results because the default test they use is affected by body mass and diet, and made even worse by e.g. preworkout supplements with creatine. None of my doctors have been aware of this, and I've had to explain it to them.

I'll preface this with I generally trust doctors. I think on the whole they are well positioned to provide massive benefit to their patients.

I will also preface this with saying I do not think any LLM is better than the average doctor and that you are far better served going to your doctor than asking ChatGPT what your health is like on any factor.

But I'll also say that the quality of doctors varies massively, and that a good amount of doctors learn what they learn in school and do not keep up with the latest advances in research, particularly those that have broad spectrums such as GPs. LLMs that search scientific literature, etc., might point you in the direction of this research that the doctors are not aware of. Or hallucinate you into having some random disease that impacts 3 out of every million people and send you down a rabbithole for months.

Unfortunately, it's difficult to resolve this without extremely good insurance or money to burn. The depth you get and the level of information that a good preventative care cardiologist has is just miles ahead of where your average family medicine practitioner is at. Statins are an excellent example - new prescriptions are for atorvastatin are still insanely high despite it being a fairly poor choice in comparison to rosuvastatin or pitavastatin for a good chunk of the people on it. They often are behind on the latest recommendations from the NLA and AHA, etc.

There's a world where LLMs or similar can empower everyday people to talk to their doctor about their options and where they stand on health, where they don't have to hope their doc is familiar with where the science has shifted over the past 5-10 years, or cough up the money for someone who specializes in it. But that's not the world of today.

In the mean time, I do think people should be comfortable being their own advocates with their doctors. I'm lucky enough that my primary care doc is open to reading the studies I send over to him on things and work with me. Or at least patient enough to humor me. But it's let me get on medications that treat my symptoms without side effects and improved my quality of life (and hopefully life/healthspan). There's also been things I've misinterpreted - I don't pick a fight with him if we come to opposite conclusions. He's shown good faith in agreeing with me where it makes sense to me, and pushed back where it hasn't, and I acknowledge he's the expert.

One challenge that may sound obvious.. is that super rare stuff gets seen super rarely, even by specalists.

In practice it means you often have to escalate from GP to local specialist to even more narrow specialist all the way to one of the regional big city specialist that almost exclusively get the weird cases.

This is because every hop is an increasingly narrow area of speciality.

Instead of just “cancer doctor” its the “GI cancer doctor” then its “GI cancer doctor of this particular organ” then its “an entire department of cancer doctors who work exclusively on this organ who will review the case together”, etc.

> I live in a place where getting a blood test requires a referral from a doctor,

To me, this is horrific. I am the advocate for my own health. I trust my doctor - he's a great guy. I have spoken to him extensively around a variety of health matters and I greatly trust his opinion.

But I also recognize that he has many other patients and by necessity has to work within the general lines of probability. There is no way for him to know every confounding and contributing factor of my health, no matter how diligent I am in filling out my chart.

I get my own bloodwork done regularly. This has let me make significant changes in my life to improve health markers. I can also get a much broader spectrum of tests done than the standard panel. This has directly lead to productive conversations with my doctor!

And from a more philosophical standpoint, this is about understanding my own body. The source of the data is me. Why should this be gatekept behind a physician referral? I find it insane to think that I could be in a position where I am not allowed to find out the cholesterol serum levels in my blood unless a doctor OKs it! What the fuck?

The issue is that whilst the warning exists and is there front and centre, the marketing around ChatGPT etc - which is absolutely deafening in volume and enthusiasm - is that they're PHD level experts and can do anything.

This marketing obscures what the software is _actually_ good at and gives users a poor mental model of what's going on under the hood. Dumping years worth of un-differentiated health data into a generic chatGPT chat window seems like a fundamental misunderstanding of the strengths of large language models.

A reasonable approach would be to try to explain what kind of tasks these models do well at and what kind of situations they behave poorly in.