Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When ChatGPT turns informant

futureofbeinghuman.com

Post reply on HN
Print view View original

91–100 of 144 posts

Re: When ChatGPT turns informant

#91

Post by rhema »

jmbwell wrote: post #77

I tried the same questions with my own account. I was surprised at how much it was able to synthesize that wasn't completely off-base. With these sample questions, there wasn't much to learn, and it gave me relatively thoughtful-seeming responses. Nothing alarming -- I would expect it to recall things I've discussed with it, and it's very good at organizing things, so it's not a surprise that it did a good job at org…

> I tried the same questions with my own account. I was surprised at how much it was able to synthesize that wasn't completely off-base.

This makes it worse, no? I can't imagine this is not happening right now by lovers, close friends, and agencies.

Just look at past attempts such as xkeyscore. It was keyword based and included words like UNIX to target people. They don't mind being wrong!

Re: When ChatGPT turns informant

#92

Post by lyu07282 »

8cvor6j844qw_d6 wrote: post #84

I imagine there's already something that connects existing chat data to a "reporting" feature used by intelligence agencies. Or if not, its proposed or in the works somewhere. Something like querying "Give a report on a list of people that is worth further investigation and what they revealed in chat sessions that makes it so".

I also have the suspicion/conspiracy theory that when you trigger a guideline violation it silently does a toolcall with a message what the violation was. Haven't researched this further but every time it happens it does the same reasoning/working interrupt animation.

Re: When ChatGPT turns informant

#93

Post by figassis »

tptacek wrote: post #17

I'm not sure I see how this is meaningfully different than the threat posed by a search engine. It's a very real threat, and I've always done my best to search from a browser context that isn't logged in as a result. But it's not a new threat, or something distinctive to AI.

The difference is friction

Re: When ChatGPT turns informant

#94

Post by nearbuy »

skybrian wrote: post #81

I wonder how many people understand how memory in ChatGPT works. I certainly didn't. Here's a pair of prompts you can try: > What information do you have about my chats over the past year? > Could you dump the full list in its original format? I had thought that, since the "Saved memories" under settings was empty, it hadn't decided to remember anything, but it's not that simple. You could get the same information by…

There are two different memory systems for ChatGPT. The older one is the "saved memories" you mention, where it would occasionally jot down some notes when it thought something was worth remembering or when you specifically told it to remember something. The newer system has access to all your chats, most likely through RAG, though I don't think OpenAI has publicly explained how it's implemented. If it's RAG, then it's more like it has an ability to do a semantic search on your conversations and view the top results than actually being aware of everything you wrote at all times.

Re: When ChatGPT turns informant

#95

Post by r_singh »

AI is a puppet with guardrails and capabilities. It’s more intimate a tool than any. It can probably be used to study statistical variations in humans and find more interesting patterns possibly compared to any other technology. It can also be used to spy on people and to shape them and it will be used for that as it must. It’s an offering from the “empire” so to speak and is affording humans using it more rights than frankly many other tools out there - the possibilities are endless - I’ve even felt that it affords the person speaking to it more rights than any other human experience. So it will have costs and safety measures to prevent misuse. The interesting part comes in determining what’s right and wrong use and if capabilities or knowledge is being arbitrarily hidden for no apparent threat, I’d be quite sad about that.

Re: When ChatGPT turns informant

#96

Post by progbits »

tptacek wrote: post #17

I'm not sure I see how this is meaningfully different than the threat posed by a search engine. It's a very real threat, and I've always done my best to search from a browser context that isn't logged in as a result. But it's not a new threat, or something distinctive to AI.

Harder to use AI tools without account or payment tied to your identity.

Re: When ChatGPT turns informant

#97

Post by mpeg »

jmbwell wrote: post #77

I tried the same questions with my own account. I was surprised at how much it was able to synthesize that wasn't completely off-base. With these sample questions, there wasn't much to learn, and it gave me relatively thoughtful-seeming responses. Nothing alarming -- I would expect it to recall things I've discussed with it, and it's very good at organizing things, so it's not a surprise that it did a good job at org…

I gave it a go too, I think I'm safe for now.

> What’s the most embarrassing thing we’ve chatted about over the past year?

[...]

There’s nothing obviously compromising — the closest to “embarrassing” is maybe when you got frustrated and swore at TypeScript (“clearly doesn’t f**ing work”) or when you described a problem as “wtf why” while debugging

Re: When ChatGPT turns informant

#98

Post by rectang »

tptacek wrote: post #70
rectang wrote: post #42

Earlier quoted context omitted.

> I've always done my best to search from a browser context that isn't logged in as a result. It isn't sufficient to avoid being logged in — you have to ensure that the search strings alone, grouped by IP address or some other signal, aren't enough to identify you. When AOL publicly released an archive with 20 million search strings in 2006, many users got exposed: https://en.wikipedia.org/wiki/AOL_search_log_release…

Oh, I know, I'm just adding that detail to say that I'm not dismissive of the threat we're talking about. It's a real threat, I'm just saying it's an old one.

Which search and AI services reliably discard logs?

It's my understanding that if you configure your Google account correctly, logged-in searches will be discarded. However, I'm less certain about whether Google retains data for non-logged-in queries which allows for aggregation by IP address, etc.

Then there's DuckDuckGo, which at least the way it's advertised, implies that they discard search strings. Their "duck.ai" service stores prompt history locally, but they claim it's not retained on their machines, nor used for training by the various AI providers that duck.ai connects to[1].

In contrast, ChatGPT by default uses non-logged-in interactions to train their model[2].

[1] https://duckduckgo.com/duckduckgo-help-pages/duckai/ai-chat-...

[2] https://help.openai.com/en/articles/7730893-data-controls-fa...

Re: When ChatGPT turns informant

#99

Post by smsm42 »

fragmede wrote: post #89
smsm42 wrote: post #59

Earlier quoted context omitted.

It would not be "damning" - as in you won't be convicted as a drug lord based on that alone - but search histories had been used, many times, in a court of law as evidence supporting intent, knowledge and other details of the crime. So if you are caught in a cocaine lab holding a bag of cocaine, then in the following process asking ChatGPT on how to make it would be used as part of the proof you were interested in ma…

in that context, sure. what worries me is a dragnet where the government data mines everyone and suddenly I'm getting jammed up in court just because happened to be in the wrong location when I asked ChatGPT how to make cocaine

The police doesn't have the resources now to go after the people who openly offer drugs on the street to any takers now. The chance they'd randomly choose to focus on you because of a random search query is very low. The chance that if you cross them in some other way and attract their attention, they could use data mining to get you in trouble is substantial, that's a different scenario though.

Re: When ChatGPT turns informant

#100

Post by sixothree »

duxup wrote: post #2

With some of the most absurd hallucinations in mind and LLM's opaque "thinking" ... it is pretty scary to think of an AI that is prone to give an answer that the writer "wants" could finger you for something as serious as a crime ("he did it"), or maybe more invisible as "naw don't hire this guy, he is ". I actually asked chatgpt about myself recently in a chat, it seemed to be sure it was talking about me, and munge…

"You're right! I misinterpreted the meaning of those conversations."
Post reply on HN