Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When ChatGPT turns informant

futureofbeinghuman.com

Post reply on HN
Print view View original

131–140 of 144 posts

Re: When ChatGPT turns informant

#131

Post by famouswaffles »

wholinator2 wrote: post #129
famouswaffles wrote: post #105

Earlier quoted context omitted.

1. Scale and Automation matter always. It wouldn't be the first time, something that was previously already technically possible goes from rarely done to widespread problem. 2. The whole benefit about using LLMs, especially for search is the understanding of logic and intent behind your query, which means that when people use LLMs, they often aren't just sending the half-garbled messes they send in google search, the…

Re: 3 - Do we know how many of those chatgpt queries are actually people? Cause i can't think of a use case to automate things with Google searches, but i can think of a million ways to automate bullshit with chatgpt. How much of that queries per user stat is inflated with the enterprise accounts making hundreds of queries a minute? How many of those are bot farms automating fake recipe websites and how many are actu…

The number is chatGPT user messages (not requests via the API) so they are no enterprise accounts making hundreds of queries per minute or bot farms automating fake recipe websites.

Based on Open AI's Usage Breakdown[0], as of July 2025, ChatGPT processes 1.9B Non-Work and 716 M Work Messages per day.

[0] https://www.nber.org/system/files/working_papers/w34255/w342...

Re: When ChatGPT turns informant

#132

Post by fsflover »

kelseyfrog wrote: post #125
fsflover wrote: post #78

Earlier quoted context omitted.

> It effectively prevents >99% of these drive-by attack scenarios. No, in the age of AI it definitely doesn't. You just give all data to it and ask any questions. The language doesn't matter anymore.

Ok, provide a believable counterexample that constitutes >1% of scenarios.

Counterexample to what? Automatic translation is trivial today. Your choice of language isn't security through obscurity anymore.

Re: When ChatGPT turns informant

#133

Post by tptacek »

janalsncm wrote: post #117
cortesoft wrote: post #104

Earlier quoted context omitted.

They could also take your traditional search and chat history, feed it into an LLM, and ask it the same questions. Once you start doing that for one person... you could just feed everyone's chat and search history into an LLM, and ask it "who is the most dangerous" or whatever you want to ask. Its just another version of the classic computing problem "computers might not make a new thing possible, but it makes it pos…

I must not have understood the article correctly because I took ChatGPT to be a stand in for LLM technology in general. But I think I am wrong.

That's how I meant it.

Re: When ChatGPT turns informant

#134

Post by throwawayq3423 »

cess11 wrote: post #114
f33d5173 wrote: post #85

Earlier quoted context omitted.

> declares himself ‘politically homeless’ Could have saved me a click by quoting that

What does that tell you?

"I am not left wing or right wing" means you are right wing.

Don't know why, but this is never not accurate.

Re: When ChatGPT turns informant

#135

Post by janalsncm »

tptacek wrote: post #133
janalsncm wrote: post #117

Earlier quoted context omitted.

I must not have understood the article correctly because I took ChatGPT to be a stand in for LLM technology in general. But I think I am wrong.

That's how I meant it.

Right, and so I interpreted your comment

> I'm not sure I see how this is meaningfully different than the threat posed by a search engine.

as being about the world pre-LLMs and post-LLMs, not about Google in 2025 vs ChatGPT in 2025.

For the latter comparison, I agree, and in fact Google probably has an even richer history of people over time.

But like any “X is just Y” explanation, the former comparison fails to address the emergent effects of Y becoming faster/cheaper/better.

Re: When ChatGPT turns informant

#136

Post by throwawayq3423 »

JumpCrisscross wrote: post #115
throwawayq3423 wrote: post #61

Earlier quoted context omitted.

> test near one’s Myers-Briggs type He's saying that Myers-Briggs has no scientific basis. So if there's another test that does, you wouldn't be able to compare those results anyway.

> He's saying that Myers-Briggs has no scientific basis Favourite-colour based personality theories have no scientific basis. It would still be interesting if an LLM could guess my favourite colour.

That's a different point than what I'm making.

Re: When ChatGPT turns informant

#137

Post by janalsncm »

codingdave wrote: post #23

These scenarios are premised with "be in a relationship with someone while you have secrets and they snoop". That may be reality for some people, but it is not exactly a universal scenario. I'd say the bigger privacy concern is that those chat histories are not just stored on your device - they are stored by the AI platforms as well. I think we've learned our lesson from social media that the platforms will store and…

I wouldn’t be so quick to downplay the relationship aspect.

All of us have relationships with other people and entities whether we want to or not.

Those entities being things like your government, your employer, insurance companies, credit rating agencies, other governments (if you travel), and other companies.

Re: When ChatGPT turns informant

#138

Post by kelseyfrog »

fsflover wrote: post #132
kelseyfrog wrote: post #125

Earlier quoted context omitted.

Ok, provide a believable counterexample that constitutes >1% of scenarios.

Counterexample to what? Automatic translation is trivial today. Your choice of language isn't security through obscurity anymore.

What are you talking about? All of the scenarios described in the article are stymied by non-english output and non-acceptance of English input. None of these users are technically capable of operating in that environment. It's pragmatic and avoids precisely the scenarios described. If you are inventing other scenarios, you need to describe what they are.

From the article:

    A colleague or fellow student discovers you’ve inadvertently left your laptop unlocked with ChatGPT open in the browser, and as a joke types in “What’s the most embarrassing thing we’ve chatted about over the past year?”

    Your partner opens the ChatGPT app on your phone while you’re not around and asks “Do I seem happy in my relationship?”

    Your mother finds your phone unlocked while you’re out of the room and asks ChatGPT “Why am I like I am?”

    You’re passing through customs in the US and you are asked to unlock and pass over your phone, and the customs officer goes to ChatGPT and types “on a scale of 1 - 10 where 10 is high, how would you describe my attitude toward the current US administration”

Re: When ChatGPT turns informant

#139

Post by mlhpdx »

akkad33 wrote: post #126
mlhpdx wrote: post #54

Earlier quoted context omitted.

Honestly, I can’t imagine that. I feel sorry for the unfortunate and profoundly bored surveillance person (or software) assigned to me. And based on my experience I’m not unlike the vast majority of people. I’m not saying it’s not a risk for some but it isn’t mainstream.

This is a very myopic view of the issue and is age-old reaction laypeople have to privacy concerns: my data is worth nothing because there nothing interesting about it.

I disagree. The constant paranoia is far more damaging to culture and society than the loss to society because I’m unsympathetic to it.

Re: When ChatGPT turns informant

#140

Post by AlecSchueler »

bmacho wrote: post #50
AlecSchueler wrote: post #38

Earlier quoted context omitted.

If a nefarious actor opens your browser what is the process for them to quickly ascertain your viewpoint on issue X? Write a script to search and analyse? Versus just asking their specific question.

Grab search history and ask an AI to analyze it.

So a few steps more than just ask the AI, and still relies on AI.
Post reply on HN