I'm less interested in what "grade" the AI gave and much more interested in what therapy or remedy it would have suggested. That's curiously lacking here.
I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
61–70 of 240 posts
Re: I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
#62Earlier quoted context omitted.
> Could you tell me which source of information do you see as "perfect" (or acceptable) that you see as a good example of a threshold for what you think the public should and should not have access to? I know it when I see it. > I don't see how this is relevant. It's relevant because blood testing is an imperfect tool. Laypeople lack the knowledge/experience to identify imperfections and are likely to take results at…
> I know it when I see it. This isn't a reasonable answer. No action can be taken and no conclusion/thought can be made from it. > Do you agree with that restriction? People should be able to perform and be informed about their own blood measurements, and possibly bring something up with their doctors outside of routine exams (which they may not even be insured for in the US). I think the restriction on medical advic…
> This isn't a reasonable answer.
Sure it is. The world runs on human judgement. If you want me to rephrase I could say that the threshold for imperfection should reflect contemporary community standards, but Stewart's words are catchier.
> I think the restriction on medical advice/conclusion, that results in treatment, is very good, otherwise you end up with "Wow, look at these results! you'll have to buy my snake oil or you'll die!".
Some people would describe this as an infringement on their free speech and bodily autonomy.
Which is to say that I think you and I agree that people in general need the government to apply some degree of restriction to medicine, we just disagree about where the line is.
But I think if I asked you to describe to me exactly where the line is you'd ultimately end up at some incarnation of "I know it when I see it".
Which is fine. Even good, I think.
> I don't believe in reducing society to a level that completely protects the most stupid of us.
This seems at odds with what you said above. A non-stupid person would seek multiple consistent opinions before accepting medical treatment, after all.
Re: I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
#63Earlier quoted context omitted.
> the onus is on them to make it reliable. That is not a plausible outcome given the current technology or of any of OpenAI's demonstrated capabilities. "If Bob's Hacksaw Surgery Center wants to stay in business they have to stop killing patients!" Perhaps we should just stop him before it goes too far?
> That is not a plausible outcome given the current technology or of any of OpenAI's demonstrated capabilities OpenAI has said that medical advice was one of their biggest use-cases they saw from users. It should be assumed they're investigating how to build out this product capability. Google has LLMs fine tuned on medical data. I have a friend who works at a top-tier US medical research university, and the universi…
They are going to do the same thing they do with code.
They are going to hire armies of developing world workers to massage those models on post-training to have some acceptable behaviors, and they will create the appropriate agents with the appropriate tools to have something that will simulate the real thing in a most plausible way.
Problem is, RLVR is cheap with code, but it can get very expensive with human physiology.
Re: I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
#64A simple understanding of transformers should be enough to make someone see that using an LLM to analyze multi-variate time series data is a really stupid endeavor.
It should be obvious to even the most dim-witted idiot with a PhD in statistics and AI
You only need this if you are a researcher. Undergraduate knowledge of Calculus and Linear Algebra is more than enough to have quite a good understanding of ML in general, and LLMs in particular.
Maybe a very small bit of Information Theory (a couple of Shannon's papers are enough) and some classical books on Natural Language Processing from the late 90s and early 2000 so you have an idea of what Language Models are outside the modern Deep Learning driven approach.
Re: I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
#65the problem with ai is that it isn’t good at recognizing red flags in data. i used it to find red flags in a financial report and it finds red flags in virtually every financial report it lays eyes on.
Re: I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
#66Earlier quoted context omitted.
> it stuck to its guns It gave you a probabilistic output. There were no guns and nothing to stick to. If you had disrupted the context with enough countervailing opinion it would have "relented" simply because the conversational probabilities changed.
It's amazing this still needs to be said, especially here
Here, sure.
For the general public, these tools have been advertised this way.
So if a good subset of HN still gets fooled, the layperson is screwed.
Re: I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
#67Health data, medical records, even research data, is very scarce in the public domain. This is not just due to so-called privacy concerns, but because such data could have generated “value” (and been sold at a good price) long before the emergence of large language models.
Re: I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
#68Health metrics are absolutely tarnished by a lack of proper context. Unsurprisingly, it turns out that you can't reliably take a concept as broad as health and reduce it to a number. We see the same arguments over and over with body fat percentages, vo2 max estimates, BMI, lactate thresholds, resting heart rate, HRV, and more. These are all useful metrics, but it's important to consider them in the proper context tha…
>I'd go so far to say this is probably the case for most people. Your average person is in really poor fitness-shape but just fine health-shape.
Modern medicine has failed to move into the era of subtlety and small problems and many people suffer as a result. Fitness nerds and general non-scientists fill the gap poorly so we get a ton of guessing and anecdotal evidence and likely a whole lot of bad advice.
Doctors won't say there's a problem until you're SICK and usually pretty late in the process when there's not a lot of room to make improvements.
At the same time, doctors won't do anything if you're 5% off optimal, but they'll happily give you a medicine that improves one symptom that's 50% off optimal that comes along with 10 side effects. Although unless you're dying or have something really straightforward wrong with you, doctors don't do much at all besides giving you a sedative and or a stimulant.
Doctors don't know what to do with small problems because they're barely studied and the people who DO try to do something don't do it scientifically.
Re: I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
#69I dunno, if the Apple Watch said he had a vo2max of 30, that probably means he can’t run a mile in less than 12 minutes or so. He’s probably not at all healthy…
Re: I let ChatGPT analyze a decade of my Apple Watch data, then I called my doctor
#70The author is a healthy person but the computer program still gave him a failing grade of F. It is irresponsible for these companies to release broken tools that can cause so much fear in real people. They are treating serious medical advice like it is just a video game or a toy. Real users should not be the ones testing these dangerous products.
> It is irresponsible for these companies I would claim that ignoring the "ChatGPT is AI and can make mistakes. Check important info." text, right under the query they type in client, is clearly more irresponsible. I think that a disclaimer like that is the most useful and reasonable approach for AI. "Here's a tool, and it's sometimes wrong." means the public can have access to LLMs and AI. The alternative, that you…
Oh I have a plan for this.
Allow it to answer general questions about health, medicine and science.
It can’t practice medicine, it can only be a talking encyclopedia that tells you how the heart works and how certain biomarkers are used. Analyzing your specific case or data is off limits.
And then when the author asks his question, it says it’s not designed to do that.