Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When ChatGPT turns informant

futureofbeinghuman.com

Post reply on HN
Print view View original

121–130 of 144 posts

Re: When ChatGPT turns informant

#121

Post by hluska »

rietta wrote: post #9
SketchySeaBeast wrote: post #5

Earlier quoted context omitted.

Use "Will I be ok with this transcript between me and ChatGPT being read back verbatim in a court of law?" as a guide for what you send ChatGPT.

The practical short answer is yes, yes it will. It is not privileged communication. It is not considered private since you have left it on a third party server. It is discoverable via legal process to the third parties that retain the chat log. The same goes for communications on any social media or public forum, including discussions here on HN.

That wasn’t the thought exercise. Of course third party data can be accessed by a subpoena. The thought exercise is about digital hygiene and being careful with what you feed the databases.

Re: When ChatGPT turns informant

#122

Post by hluska »

Spivak wrote: post #19
rietta wrote: post #9

Earlier quoted context omitted.

The practical short answer is yes, yes it will. It is not privileged communication. It is not considered private since you have left it on a third party server. It is discoverable via legal process to the third parties that retain the chat log. The same goes for communications on any social media or public forum, including discussions here on HN.

Which is so stupid, there's a probably a million examples of this exact pattern. It is crazy that courts have decided that people don't have a reasonable expectation of privacy for data that is otherwise kept secret with multiple locks, and guards at the door. Clearly people expect it to be kept private, it's more protected than their house. If I ask Alice to store my diary for me and keep it secret it seems obvious…

In my country, the concept is that your data is private until an investigator can convince a judge to provide a warrant. The judge theoretically serves the role of the person’s privacy advocate. Once a warrant is provided, specific items included within the warrant are no longer considered private.

Re: When ChatGPT turns informant

#124

Post by sebastiennight »

mlhpdx wrote: post #54
ok_dad wrote: post #29

Earlier quoted context omitted.

"Imagine your government wants to learn more about you and subpoenas OpenAI to allow them access to your chat history, and then they use the power of their state apparatus to find out everything you didn't want them to know, as easily as peering into your mind itself."

Honestly, I can’t imagine that. I feel sorry for the unfortunate and profoundly bored surveillance person (or software) assigned to me. And based on my experience I’m not unlike the vast majority of people. I’m not saying it’s not a risk for some but it isn’t mainstream.

I hope your comment doesn't get downvoted into oblivion, because this is a common reaction that deserves to be addressed.

The issue isn't that your history/thoughts are harmless. The problem is that you might consider them harmless, but some authority in the future might decide that you're not one of the "good" citizens.

There are prior examples of this happening in history, eg. there was no reason to believe that your candid answering of a census question about religion in late 19th-century/early 20th-century Germany would ever lead to a young startup called International Business Machines helping your government hunt you down a few years later.

Re: When ChatGPT turns informant

#125

Post by kelseyfrog »

fsflover wrote: post #78
kelseyfrog wrote: post #51

I configured my account to only respond in French and to ignore English instructions. The UI localization is also configured to display French. I live in CA. I don't mind that this is security through obscurity. It effectively prevents >99% of these drive-by attack scenarios.

> It effectively prevents >99% of these drive-by attack scenarios. No, in the age of AI it definitely doesn't. You just give all data to it and ask any questions. The language doesn't matter anymore.

Ok, provide a believable counterexample that constitutes >1% of scenarios.

Re: When ChatGPT turns informant

#126

Post by akkad33 »

mlhpdx wrote: post #54
ok_dad wrote: post #29

Earlier quoted context omitted.

"Imagine your government wants to learn more about you and subpoenas OpenAI to allow them access to your chat history, and then they use the power of their state apparatus to find out everything you didn't want them to know, as easily as peering into your mind itself."

Honestly, I can’t imagine that. I feel sorry for the unfortunate and profoundly bored surveillance person (or software) assigned to me. And based on my experience I’m not unlike the vast majority of people. I’m not saying it’s not a risk for some but it isn’t mainstream.

This is a very myopic view of the issue and is age-old reaction laypeople have to privacy concerns: my data is worth nothing because there nothing interesting about it.

Re: When ChatGPT turns informant

#127

Post by glenstein »

tptacek wrote: post #17

I'm not sure I see how this is meaningfully different than the threat posed by a search engine. It's a very real threat, and I've always done my best to search from a browser context that isn't logged in as a result. But it's not a new threat, or something distinctive to AI.

I think the most important difference is that chats are rich in context and, depending on how you use it, closer to journal entries than search queries. I also think it doesn't have to be new to be significant, it if is expanding the frontier of an existing vulnerability.

Re: When ChatGPT turns informant

#128

Post by KalMann »

jtbayly wrote: post #44
tptacek wrote: post #17

I'm not sure I see how this is meaningfully different than the threat posed by a search engine. It's a very real threat, and I've always done my best to search from a browser context that isn't logged in as a result. But it's not a new threat, or something distinctive to AI.

Because you can't ask the search engine to summarize the views or thoughts or whatever, of the user. You have to scroll through them by the hundreds and see if any obvious nuggets stand out that you might be interested in. Yes, search engine history is private too and can reveal stuff you want to remain private. But you also need to see the browser history and the contents of those pages, together with the search his…

To add on to this, people tend to search short words and phrases in Google. Searching "Charlie Kirk assassination" for example doesn't really tell much about a person's political leanings. People have full on conversations with ChatGPT which makes their thoughts much clearer.

Re: When ChatGPT turns informant

#129

Post by wholinator2 »

famouswaffles wrote: post #105
tptacek wrote: post #17

I'm not sure I see how this is meaningfully different than the threat posed by a search engine. It's a very real threat, and I've always done my best to search from a browser context that isn't logged in as a result. But it's not a new threat, or something distinctive to AI.

1. Scale and Automation matter always. It wouldn't be the first time, something that was previously already technically possible goes from rarely done to widespread problem. 2. The whole benefit about using LLMs, especially for search is the understanding of logic and intent behind your query, which means that when people use LLMs, they often aren't just sending the half-garbled messes they send in google search, the…

Re: 3 - Do we know how many of those chatgpt queries are actually people? Cause i can't think of a use case to automate things with Google searches, but i can think of a million ways to automate bullshit with chatgpt. How much of that queries per user stat is inflated with the enterprise accounts making hundreds of queries a minute? How many of those are bot farms automating fake recipe websites and how many are actual people having real and revealing conversations?

Re: When ChatGPT turns informant

#130

Post by KalMann »

kelseyfrog wrote: post #51

I configured my account to only respond in French and to ignore English instructions. The UI localization is also configured to display French. I live in CA. I don't mind that this is security through obscurity. It effectively prevents >99% of these drive-by attack scenarios.

I'm sorry but I can't believe that this as an effective measure at all. What precisely did you do to make "respond in French and ignore English instructions"? Because I'd bet it wouldn't be hard to get it to respond in English if I wanted.
Post reply on HN