Earlier quoted context omitted.
If it makes a mistake? You’re not required to follow the AI, just use it as a tool for consideration.
Doesn't sound very $1 trilliony
Every valuable industry is rife with errors.
ChatGPT terms disallow its use in providing legal and medical advice to others
421–430 of 439 posts
Re: ChatGPT terms disallow its use in providing legal and medical advice to others
#422Re: ChatGPT terms disallow its use in providing legal and medical advice to others
#423Earlier quoted context omitted.
This is what really scares me about people using AI. It will confidently hallucinate studies and quotes that have absolutely no basis in reality, and even in your own field you're not going to know whether what it's saying is real or not without following up on absolutely every assertion. But people are happy to completely buy its diagnoses of rare medical conditions based on what, exactly?
Give a single example using gpt-5 thinking.
GPT-5 thinking is one of the biggest offenders and it's quite incredible to me that you think it doesn't hallucinate. It makes me strongly suspect your own judgment is impaired. Also, what do you mean asking for "reproducible examples"? Is it somehow not a valid example if it only sometimes makes up citations?
Re: ChatGPT terms disallow its use in providing legal and medical advice to others
#424Earlier quoted context omitted.
Give a single example using gpt-5 thinking.
GPT-5 thinking is one of the biggest offenders and it's quite incredible to me that you think it doesn't hallucinate. It makes me strongly suspect your own judgment is impaired. Also, what do you mean asking for "reproducible examples"? Is it somehow not a valid example if it only sometimes makes up citations?
Share any example where I can get it to make an obvious error
Re: ChatGPT terms disallow its use in providing legal and medical advice to others
#425Grok will do mostly whatever you want. It's a tool and it's your personal responsibility to use it correctly. That's the way it should be. You don't need Sam Altman to babysit you, I hope.
> It's a tool and it's your personal responsibility to use it correctly. Sure in theory, but this never works out in practice when applied to the populace. Replace "LLM" with other controversial things — social media, cigarettes, guns. I'm also certain anyone outside the tech sphere who takes personal responsibility when using LLMs is not using Grok as their daily driver.
Grok Code Fast 1 is quite usable for small simple coding tasks. And quite a bit cheaper than say: Claude Sonnet 4.5
Personal experience when using it OpenRouter.
That said, I prefer bigger models even if they cost more. But I do this through GitHub Copilot at a steady 10 dollars per month. I can do way more request with it than I can with 10 dollars worth of credits at OpenRouter. I don't think Copilot is making any money, on the contrary.
Re: ChatGPT terms disallow its use in providing legal and medical advice to others
#426Earlier quoted context omitted.
> Next will be: you're allowed to use it, if your company/product is "HippocraticGPT"-certified, a bit like SOC or PCI compliance. That's exactly how it should be. That's why one needs a license to practice medicine. I work with a company in this space. All the AI tools are for the professional to use - the patient never interacts with them. The inferred output (I prefer the word prediction) is only ever seen by a me…
Im working on a PhD concerning LLMs and somatic medicine, as an MD, and I must admit that my perspective is the complete opposite. Medical care, at the end of the day, has nothing to do with having a license or not. Its about making the correct diagnosis, in order to administer the correct treatment. Reality does not care about who (or what) made a diagnosis, or how the antibiotic you take was prescribed. You either…
I appreciate your perspective. May I contact you? You are invited to send me an email, my Gmail username is the same as my HN username.
Re: ChatGPT terms disallow its use in providing legal and medical advice to others
#427Earlier quoted context omitted.
I would expect that to work exactly how I code with it. I would examine the inferred output, decide if it is fit for purpose, and if so then commit it. If not, I would either refine my prompt or make the fixes to the inferred code myself. Would this hypothetical legal professional not verify that the inferred output (which is really a prediction) is correct?
As an attorney, I don't see what the value of that would be if I have to double check its work. How else could I verify that the output is correct?
The value is, at least in my field, that checking for correctness is often a less tiring task than writing the code. With proper prompting, the LLM will also cover more corner cases than will a human think to cover. And to be honest, I really like the names of internal identifiers that LLM come up with. It's a skill that is notoriously lacking in software development.
Additionally, code review with the proper tools can be done relatively quickly and it's always a good idea to get a second opinion - even that of an LLM. I suppose that the human could write the code then ask the LLM for a code review - but that is not common practice.
Re: ChatGPT terms disallow its use in providing legal and medical advice to others
#428Earlier quoted context omitted.
this is fresh news right? a friend just used chatgpt for medical advice last week (stuffed his wound with antibiotics after motorbike crash). are you saying you completely treated the congenital issue in this timeframe?
He’s simply saying that ChatGPT was able to point them in the right direction after 1 chat exchange, compared to doctors who couldn’t for a long time. Edit: Not saying this is the case for the person above, but one thing that might bias these observations is ChatGPT’s memory features. If you have a chat about the condition after it’s diagnosed, you can’t use the same ChatGPT account to test whether it could have diag…
This was before I started using it as anything but a toy. I had definitely not spoken about medical issues; it was on a whim.
Re: ChatGPT terms disallow its use in providing legal and medical advice to others
#429Earlier quoted context omitted.
Again, it's not that all telehealth doctors are great at this, it's that LLMs are awful at caving in to saying something with warnings the reader will opt to ignore instead of being adamant things are just too uncertain to say anything of value when continually prompted. This is largely because an LLM guessing an answer is rewarded more often than just not answering, which is not true in the healthcare profession.
I follow the logic, I'm just not sure the claim is right.
LLMs almost never reply with I don’t know. There’s been mountains of research as to why this is, but it’s very well documented behavior.
Even in the rare case where an LLM does reply with I don’t know go see your doctor, all you have to do is ask it again until you get a response you want.
Re: ChatGPT terms disallow its use in providing legal and medical advice to others
#430Earlier quoted context omitted.
This isn’t accurate though, the models do push back and do question. I would say ChatGPT is way way better than the average therapist. I’ve seen maybe between 15-20 psychotherapists over the course of my life and ChatGPT is better than 85% of them I would wager. I’ve had a therapist recommend me take a motorbike trek across Europe (because that’s what he did) to cure my depression. I think people tend to radically ov…
> I’ve had a therapist recommend me take a motorbike trek across Europe (because that’s what he did) to cure my depression. When I try to help people in support group/ therapy group although I am not a therapist, I also try to explain how I fixed my issues or how I do so. I feel like, that isn't bad take to be really honest in the sense that I personally feel like there are times when we lose the fact that our lives…
But telling someone living in New York with a full-time job and a girlfriend etc, that the solution to their depression is to quit their job and take several months to travel across Europe on a motorbike isn't exactly practical advice. Which was my point.
I could get better advice by asking an LLM what to do about my depression.