We need to tell people ChatGPT will lie to them, not debate linguistics

That's a great distinction to draw.

I think it will split in two. There will be cases where the LLM has the truth represented in its data set and still chooses to say something else because its training has told it to produce the most plausible sounding answer, not the one closest to the truth. So this will fit closer to the idea of real lying.

A good example: I asked it what the differences in driving between Australia and New Zealand are. It confidently told me that in New Zealand you drive on the right hand side of the road while in Australia you drive on the left. I am sure it has the correct knowledge in its training data. It chose to tell me that because that is a more common answer people say when asked about driving differences because that is the more dominant difference when you look between different countries.

Then there will be cases where the subject in question has never been represented in its data set. Here I think your point is very valid.

The point is to not mislead people about what is going on, of course.

>>How is this not “paternalistic”?

I see three categories of handling this issue, perhaps call them "Paternalistic", "Responsible", and "Anti-Social".

- Paternalistic would be like: "We assess [thing] to be dangerous in a number of ways and so we forbid you to use [thing], except by using our high priesthood representatives as intermediaries".

- Responsible would be like: We assess [thing] to be dangerous in a number of ways, so we clearly inform you that [thing] is dangerous, specify the risks, and how to avoid or mitigate them. You are then free to use [thing] as you see fit. (Note: if using [thing] badly causes public as well as private risks, it is responsible to require training, tests, and licensing to use [thing] in public).

- Anti-Social would be like: We know [thing] is dangerous, but we DGAF what happens, you should figure it out all by yourself, and if you have a problem, piss-off. Examples: giving a CRT monitor to a novice hardware hacker who has only ever seen LED monitors and not telling her in detail about the high-voltage capacitor hazards inside or how to deal with them. Giving a gun to someone without the basic rules (always treat it as if it's loaded, never point it at anything you do not intend to destroy, trigger finger discipline, etc.). A mere "Hey, watch it with that thing" is inadequate, and you'd rightly bear responsibility when they electrocute themselves, shoot themselves in the foot, etc.

A decent warning around LLMs could be: "Be warned: while this system can produce amazing and useful results it will frequently lie to you, and with great confidence. Be sure to check all results independently. Do NOT USE IT for any life-critical or potentially life-changing decisions. E.g., Do NOT use it as a sole source for medical diagnoses, as it may give a misdiagnosis that would kill you [0]. Do NOT use it as a sole source for therapy, as it may counsel you to suicide [1]. These are real harms, do NOT use this as the sole source of any answers." - - - Then, let them loose on it. They know about the footguns, and have been warned

Anything less borders on anti-social.

>> because they will revert to cannibalism if not properly informed

It is not that they'll revert to cannibalism if not properly informed, it is that there are potential and serious harms that are easily avoided if they are told.

The civilized thing to do when there is a bridge out ahead leading the road into raging floodwaters, is to tell your fellow travelers before they get there, not merely expect that every one of them will notice in time and handle it well.

What is a given among civilized people is that if we know something has hazards, we give warnings and information to our fellow travelers. That's not paternalistic. Paternalistic is to not give warnings, but to forbid use. And it is uncivilized and anti-social to fail to take the trouble to give full and straight information.

Informed consent is a good thing. Handing somebody what is effectively a booby-trap without telling them is not.

[0] https://inflecthealth.medium.com/im-an-er-doctor-here-s-what...

[1] https://interestingengineering.com/culture/belgian-woman-bla...

Wow, great research.

I didn't say "regulation". You can encourage norms in the model building stages. You can encourage norms in all sorts of places. The industry can certainly adopt "standards" or "best practices" not matter how fast the industry thinks it is moving.