I’m a doctor: Here’s what I found when I asked ChatGPT to diagnose my patients

Before ChatGPT there wasn't anything quite like it, so I think that the goalposts were mostly about passing the Turing test, or at least "not sucking" after 4 or 5 phrases. The leap from whatever existed before it was enormous, and I think comparable to the first iPhone versus, let's say, the Blackberry or Windows CE. Maybe even more than that. Go back to the first iPhone, it had a lot of issues but it offered so much as it was that people wouldn't look back.

It's too much to expect ChatGPT 3 to be better an experienced emergency doctor or neurosurgeon. But the fact that it can be a competent assistant to a doctor instead of a toy assistant - that by itself is amazing.

Most tech support is little more than Googling the right question and going through the steps in the first or second result. Knowing what questions to Google and what answers won't apply is the reason you get paid for that stuff.

I, for one, like my doctor to use tools to find possible diagnoses that she may have learned about 30 years ago but rarely ever come up, as long as the tools they use preserve my privacy.

I mean, if that was ever the goalpost, ChatGPT does not clear it: the Turing test isn't about merely tricking someone who isn't paying attention, or making someone seriously doubt their belief system... it involved a careful discussion with a chat agent by someone who actively was trying to determine whether or not it/they were human. If you sit down with ChatGPT--even 4--and start discussing complex topics about the world with it, you absolutely can quickly get it starting to adamantly defend things about the world which a human would never assert... and, even more damning, if you just ask it "are you human?" it will reply "no", because we actively told it to fail the Turing test.

If we instead merely re-define Turing's test to be "it is able to converse with a human and manages to occasionally trick a human into thinking it is also human" then I had a friend (Roy Keene) who managed to pull that off years ago with a chatbot that was literally just "every time someone messages it, wait a small random period of time and then reply with a random response from the following set: Yes, No, Maybe, I dunno, :), :(, ;P (and like one other thing I forget)". He had it set up to reply when he was not available and when I finally caught him (I asked about a task I gave him the day before which he claimed to have no memory of, so he admitted to me he had this bot) he gave me a log file and OMG I had had long conversations about personal topics with this thing and had never noticed... but only because I wasn't trying.

You can send it off on all sorts of legwork research missions but don’t expect perfect results and sometimes you’ll get crazy ones. Used the right way, if you are already an expert in the field or knowledgeable and able editor , that can save a whole lot of time. But taken verbatim it is anywhere from ok to dangerous.

Separately, the models’ skills with natural language are clear and impressive, but it seems like they need to be coupled with a deterministic knowledge representation system for suitable reasoning.

Perhaps the abilities of these models to ingest large amounts of text could be used to enhance / create such representation.

Cyc where are you?