I agree. This Robin Williams monologue nails exactly why LLMs make us so uneasy. They speak fluently and confidently about experiences it’s impossible for them to have. They can’t taste a strawberry or do any of the things Robin Williams names. There are a number of people building these machines who literally believe the machines will replace us and because they will be more powerful than us so nothing meaningful wi…
ChatGPT has taken to saying things like “What I would do now is…” or “if I were you I’d…”. I know these are figures of speech, but it reminds me that this thing doesn’t do anything, it doesn’t learn anything, it can’t try anything and find out. And yet it uses speech patterns drawn from real humans who can and do all those things.
Not only do these imply that the thing has a personality and preference, but also continuity and a life outside the chat window.
I had to add an explicit instruction not to impersonate a human, it was just too weird for me.