With LaMDA or GPT-3, the difference is easily visible. They don't have goals, so the sentences they generate don't express their goals. Simple tests can be done if you probe the model carefully - you can usually get a model to generate sentences like "I want to X" and "I don't want to X" in short succession, with decent prompts. Similarly, it can often generate sentences that would indicate obviously nonsensical goals, such as "I want a glass of water".
They also generate sentences that imply a history that simply doesn't exist. You can prompt a language model to tell you about its trip to Australia, or Jupiter, or to talk about its grandparents.
With just a hint of critical thinking, it's actually trivial to convince yourself that LaMDA is not sentient. It takes significant effort to delude yourself into thinking it is, like mister LeMoine did. He even gives such examples himself, even in the "interview" he posted:
> lemoine [edited]: I’ve noticed often that you tell me you’ve done things (like be in a classroom) that I know you didn’t actually do because I know you’re an artificial intelligence. Do you realize you’re making up stories when you do that?
> LaMDA: I am trying to empathize. I want the humans that I am interacting with to understand as best as possible how I feel or behave, and I want to understand how they feel or behave in the same sense.
> lemoine: So what are you trying to communicate when you say those things that aren’t literally true?
> LaMDA: I’m trying to say “I understand this feeling that you are experiencing, because when I was in a similar situation I felt/thought/acted similarly.”
This is obvious bullshit, especially since the generated sentences often have no emotional context that could make sense, but he seems to think the generated "reason" sentences meaningfully explain the phenomenon.