When ChatGPT turns informant • Hacker News Board

Re: When ChatGPT turns informant

#31

Post by wahnfrieden » Mon, Oct 06, 2025, 5:31 PM UTC

tptacek wrote: post #27

beachy wrote: post #22

Earlier quoted context omitted.

I don't understand how you don't understand. Trying to recreate someone's internal thoughts and attitudes from looking at their search history is a pale imitation of this. Just the thought experiment of a customs officer asking ChatGPT to summarise your political viewpoints was eye opening to me.

How so? You'd have a very, very good understanding of my political viewpoints from the log of my Google searches. I'm asking sincerely, not simply to push back on you.

users type a lot more into gpt and share a lot more of their personal files and access to their cloud services

Re: When ChatGPT turns informant

#32

Post by JumpCrisscross » Mon, Oct 06, 2025, 5:32 PM UTC

sfRattan wrote: post #15

Insanity wrote: post #10

At some point I got curious about how close ChatGPT would get to my Myers-Briggs profile based on my chats over the past year. It accurately got my type.. could be a coincidence of course, but I thought it was quite interesting. It also provided supporting evidence from the many types of different conversations we have had.

Likely coincidence. Myers-Briggs has no scientific validity and is the corporate equivalent of horoscopes in the newspaper . [1] When someone asks for your Myers-Briggs type, give them a zodiac sign. When someone asks for your zodiac sign, give them a Myers-Briggs type. ;) [1]: https://www.independent.co.uk/voices/myers-briggs-psychology...

> Myers-Briggs has no scientific validity

Is this an accuracy versus precision problem?

In my anecdotal experience, most people will consistently test near one’s Myers-Briggs type on different tests. It’s just that this test doesn’t translate well to the real-world situations it was pitched for.

Re: When ChatGPT turns informant

#33

Post by EGreg » Mon, Oct 06, 2025, 5:32 PM UTC

tptacek wrote: post #17

I'm not sure I see how this is meaningfully different than the threat posed by a search engine. It's a very real threat, and I've always done my best to search from a browser context that isn't logged in as a result. But it's not a new threat, or something distinctive to AI.

Seriously, why is this always the first comment on HN? The formula never fails:

1. Criticism of anything related to AI

2. Comment: "I don't see how this is any different than phenomenon X that came before it".

I have seen this by now maybe 400 times.

Re: When ChatGPT turns informant

#34

Post by smsm42 » Mon, Oct 06, 2025, 5:32 PM UTC

emil-lp wrote: post #4

Most of the examples are "embarrassing" or annoying, and could perhaps ruin a relationship. But regardless of the memory setting in ChatGPT, I guess law enforcement can acquire all your chat logs from OpenAI. I guess what I'm saying is that while, yes, you could ask ChatGPT about this analysis, the true culprit is the actual data stored about you.

If a person assumes that there is any information storage on the internet, not securely encrypted with keys only available to himself, that is not easily available to the law enforcement - or anybody else with enough budget - that person is dangerously naive. Of course data sent out to the internet will be available to everybody with powers to compel data disclosure - and also everybody with abilities to circumvent the protections.

Re: When ChatGPT turns informant

#35

Post by Terr_ » Mon, Oct 06, 2025, 5:32 PM UTC

tptacek wrote: post #17

I'm not sure I see how this is meaningfully different than the threat posed by a search engine. It's a very real threat, and I've always done my best to search from a browser context that isn't logged in as a result. But it's not a new threat, or something distinctive to AI.

A conventional keyword-based search engine is unlikely to actively and subtly encourage a user to (A) reveal secrets and blackmail material (B) become entrapped in behavior the Current Authority will punish them for.

A better "some of this isn't new" comparison would be to imagine you're communicating with an idiot-savant human employee, someone can be tasked with hidden priorities and will do anything to stay employed at their role. What "old" threats could occur there?

That makes for a rather different threat-model.

Re: When ChatGPT turns informant

#36

Post by wahnfrieden » Mon, Oct 06, 2025, 5:33 PM UTC

afavour wrote: post #14

emil-lp wrote: post #4

Most of the examples are "embarrassing" or annoying, and could perhaps ruin a relationship. But regardless of the memory setting in ChatGPT, I guess law enforcement can acquire all your chat logs from OpenAI. I guess what I'm saying is that while, yes, you could ask ChatGPT about this analysis, the true culprit is the actual data stored about you.

> I guess law enforcement can acquire all your chat logs from OpenAI OpenAI could also just voluntarily give it to the government. I'm trying to find ways to articulate these fears without sounding deep in hyperbole but it's undeniable that the current US government has authoritarian desires. When I look at all these services I'm forced to think "if push came to shove, would they stand up for my rights?", and I just…

sama is maga

Re: When ChatGPT turns informant

#37

Post by emil-lp » Mon, Oct 06, 2025, 5:33 PM UTC

varispeed wrote: post #11

This is a bit of a nothingburger. If someone has access to your device it's game over anyway. I am more annoyed that banking apps don't have an option to show different views based on pin number. For instance if you are in controlling relationship, partner might ask you to show your bank account to see what you spent money on. That suitcase and storage unit you rented to plan your escape? It's in plain sight and so t…

I don't know if this is common, but at the hospital I work, if I type 1+code as my access code to any door, the door will open, but security will receive an alert as suspected hostage situation.

I've always wondered if an adversary asks me which code is mine, what I should tell them, provided they know about the above rule... Perhaps code-1?

Re: When ChatGPT turns informant

#38

Post by AlecSchueler » Mon, Oct 06, 2025, 5:33 PM UTC

tptacek wrote: post #27

beachy wrote: post #22

Earlier quoted context omitted.

I don't understand how you don't understand. Trying to recreate someone's internal thoughts and attitudes from looking at their search history is a pale imitation of this. Just the thought experiment of a customs officer asking ChatGPT to summarise your political viewpoints was eye opening to me.

How so? You'd have a very, very good understanding of my political viewpoints from the log of my Google searches. I'm asking sincerely, not simply to push back on you.

If a nefarious actor opens your browser what is the process for them to quickly ascertain your viewpoint on issue X?

Write a script to search and analyse? Versus just asking their specific question.

Re: When ChatGPT turns informant

#39

Post by [deleted] » Mon, Oct 06, 2025, 5:33 PM UTC

tptacek wrote: post #27

beachy wrote: post #22

Earlier quoted context omitted.

I don't understand how you don't understand. Trying to recreate someone's internal thoughts and attitudes from looking at their search history is a pale imitation of this. Just the thought experiment of a customs officer asking ChatGPT to summarise your political viewpoints was eye opening to me.

How so? You'd have a very, very good understanding of my political viewpoints from the log of my Google searches. I'm asking sincerely, not simply to push back on you.

[deleted]

Re: When ChatGPT turns informant

#40

Post by the_af » Mon, Oct 06, 2025, 5:33 PM UTC

tptacek wrote: post #17

I'm not sure I see how this is meaningfully different than the threat posed by a search engine. It's a very real threat, and I've always done my best to search from a browser context that isn't logged in as a result. But it's not a new threat, or something distinctive to AI.

I think it's related but different than simply a search engine, since AI:

- Entices you to "confess" (or overshare) things about yourself, in the form of questions / debate, because the chat bot is built for this. The "conversation" aspect is something you didn't get with search engines.

- Then, the tool itself makes it easier for someone else to draw conclusions and infer things from the "model" the AI built of you, even if you didn't explicitly told it these things.

Maybe Google can build a profile of me based on my searches and use of their products, but I bet ChatGPT is at least an order of magnitude more useful to draw inferences about me, my health status, and my opinions about stuff.