Live data from Hacker News

I caught Google Gemini using my data and then covering it up

unbuffered.stream

71–80 of 87 posts

Re: I caught Google Gemini using my data and then covering it up

#71
post #13

This is a LLM directly, purposefully lying, i.e. telling a user something it knows not to be true. This seems like a cut-and-dry Trust & Safety violation to me. It seems the LLM is given conflicting instructions: 1. Don't reference memory without explicit instructions 2. (but) such memory is inexplicably included in the context, so it will inevitably inform the generation 3. Also, don't divulge the existence of user-…

Let's stop anthropomorphizing these tools. They're not "purposefully lying", or "know" anything to be true.

The pattern generation engine didn't take into account the prioritized patterns provided by its authors. The tool recognized this pattern in its output and generated patterns that can be interpreted as acknowledgement and correction. Whether this can be considered a failure, let alone a "Trust & Safety violation", is a matter of perspective.

Re: I caught Google Gemini using my data and then covering it up

#74
I believe every AI company does this. we have proof that Google does, that Antropic does too.

and I have my own experience with OpenAI, where their chatbot referenced one of my computers having certain specs, but I mentioned those in a different log, and that information was never added to the memory.

https://chatgpt.com/share/691c6987-a90c-8000-b02f-5cddb01d01...

Re: I caught Google Gemini using my data and then covering it up

#75
post #74

I believe every AI company does this. we have proof that Google does, that Antropic does too. and I have my own experience with OpenAI, where their chatbot referenced one of my computers having certain specs, but I mentioned those in a different log, and that information was never added to the memory. https://chatgpt.com/share/691c6987-a90c-8000-b02f-5cddb01d01...

This is an advertised feature of ChatGPT, and you can switch it off if you want. https://help.openai.com/en/articles/11146739-how-does-refere...

Re: I caught Google Gemini using my data and then covering it up

#79
post #71
post #13

This is a LLM directly, purposefully lying, i.e. telling a user something it knows not to be true. This seems like a cut-and-dry Trust & Safety violation to me. It seems the LLM is given conflicting instructions: 1. Don't reference memory without explicit instructions 2. (but) such memory is inexplicably included in the context, so it will inevitably inform the generation 3. Also, don't divulge the existence of user-…

Let's stop anthropomorphizing these tools. They're not "purposefully lying", or "know" anything to be true. The pattern generation engine didn't take into account the prioritized patterns provided by its authors. The tool recognized this pattern in its output and generated patterns that can be interpreted as acknowledgement and correction. Whether this can be considered a failure, let alone a "Trust & Safety violatio…

IMHO the terms are fine, even if applied to much dumber systems, and most people will and do use the terms that way colloquially so there's no point fighting it. A Roomba can "know" where the table is. An automated voice recording or a written sign can "lie" to you. One could argue the lying is only done by the creator of the recording/sign - but then what about a customer service worker who is instructed to lie to customers by their employer? I think both the worker and employer could be said to be lying.

Re: I caught Google Gemini using my data and then covering it up

#80
post #75
post #74

I believe every AI company does this. we have proof that Google does, that Antropic does too. and I have my own experience with OpenAI, where their chatbot referenced one of my computers having certain specs, but I mentioned those in a different log, and that information was never added to the memory. https://chatgpt.com/share/691c6987-a90c-8000-b02f-5cddb01d01...

This is an advertised feature of ChatGPT, and you can switch it off if you want. https://help.openai.com/en/articles/11146739-how-does-refere...

Does the switch work though?
Post reply on HN