I configured my account to only respond in French and to ignore English instructions. The UI localization is also configured to display French. I live in CA. I don't mind that this is security through obscurity. It effectively prevents >99% of these drive-by attack scenarios.
When ChatGPT turns informant
51–60 of 144 posts
Re: When ChatGPT turns informant
#52Earlier quoted context omitted.
I don't understand how you don't understand. Trying to recreate someone's internal thoughts and attitudes from looking at their search history is a pale imitation of this. Just the thought experiment of a customs officer asking ChatGPT to summarise your political viewpoints was eye opening to me.
How so? You'd have a very, very good understanding of my political viewpoints from the log of my Google searches. I'm asking sincerely, not simply to push back on you.
It seems fairly easy to figure this out with a little thought…
When talking to a chatbot you're likely to type more words per query, as a simple measure. But you're also more likely to have to clarify your queries with logic and intent — to prevent it going off the rails — revealing more about the intentions behind your searches than just stringing together keywords.
It'd be harder to claim purely informational reasons for searching if your prompts betray motive.
Re: When ChatGPT turns informant
#53Earlier quoted context omitted.
How so? You'd have a very, very good understanding of my political viewpoints from the log of my Google searches. I'm asking sincerely, not simply to push back on you.
If a nefarious actor opens your browser what is the process for them to quickly ascertain your viewpoint on issue X? Write a script to search and analyse? Versus just asking their specific question.
Google completely owns most people's browsers, and the government has made it clear that they do not care.
Re: When ChatGPT turns informant
#54These scenarios are premised with "be in a relationship with someone while you have secrets and they snoop". That may be reality for some people, but it is not exactly a universal scenario. I'd say the bigger privacy concern is that those chat histories are not just stored on your device - they are stored by the AI platforms as well. I think we've learned our lesson from social media that the platforms will store and…
"Imagine your government wants to learn more about you and subpoenas OpenAI to allow them access to your chat history, and then they use the power of their state apparatus to find out everything you didn't want them to know, as easily as peering into your mind itself."
Honestly, I can’t imagine that. I feel sorry for the unfortunate and profoundly bored surveillance person (or software) assigned to me. And based on my experience I’m not unlike the vast majority of people. I’m not saying it’s not a risk for some but it isn’t mainstream.
Re: When ChatGPT turns informant
#55At some point I got curious about how close ChatGPT would get to my Myers-Briggs profile based on my chats over the past year. It accurately got my type.. could be a coincidence of course, but I thought it was quite interesting. It also provided supporting evidence from the many types of different conversations we have had.
Likely coincidence. Myers-Briggs has no scientific validity and is the corporate equivalent of horoscopes in the newspaper . [1] When someone asks for your Myers-Briggs type, give them a zodiac sign. When someone asks for your zodiac sign, give them a Myers-Briggs type. ;) [1]: https://www.independent.co.uk/voices/myers-briggs-psychology...
Myers-Briggs is pattern recognition that gives a surface-level measurement of 4 traits. The weakness of the test is not that it cannot produce an answer, it is that the answer is useless. It changes day-to-day as people change day-to-day and it over-simplifies people across 4 traits. But ChatGPT can work from a static history of chats, avoiding change (assuming you are just asking once). And it excels at such surface-level categorization of content.
So even if MBTI is useless, the ability of ChatGPT to offer an equally useless answer of your type is not at all surprising.
Re: When ChatGPT turns informant
#56Earlier quoted context omitted.
I don't understand how you don't understand. Trying to recreate someone's internal thoughts and attitudes from looking at their search history is a pale imitation of this. Just the thought experiment of a customs officer asking ChatGPT to summarise your political viewpoints was eye opening to me.
How so? You'd have a very, very good understanding of my political viewpoints from the log of my Google searches. I'm asking sincerely, not simply to push back on you.
(Not op)
Maybe not you in particular, but I expect people to be more forthcoming in their writing towards LLMs vs a raw google search.
For example, a search of "nice places to live in" vs "I'm considering moving from my current country because I think I'm being politically harassed and I want to find nice places to live that align with my ideology of X, Y, Z".
I do agree that, after collecting enough search datapoints, one could piece together the second sentence from the first, and that this is more akin to a new instance of an already existing issue.
It's just that, by default I expect more information to be obtainable, more easily, from what people write to an LLM vs a search box.
Re: When ChatGPT turns informant
#57I'm not sure I see how this is meaningfully different than the threat posed by a search engine. It's a very real threat, and I've always done my best to search from a browser context that isn't logged in as a result. But it's not a new threat, or something distinctive to AI.
The level of time and effort being so low increases the likelihood of this happening. It's the same sort of reason there's red teaming for ensuring AI doesn't help bad actors with chemical weapons, lowered barriers for bad things is a concern even if the bad things were possible before.
Re: When ChatGPT turns informant
#58Earlier quoted context omitted.
Seriously, why is this always the first comment on HN? The formula never fails: 1. Criticism of anything related to AI 2. Comment: "I don't see how this is any different than phenomenon X that came before it". I have seen this by now maybe 400 times.
Turns out, we're the stoichastic parrots all along!
And suddenly 8 billion people cried out "I am unique and special" in unison.
Re: When ChatGPT turns informant
#59If me asking "how do I make cocaine", and getting a refusal from ChatGPT, as if that's what was stopping me from becoming the next Pablo Escobar, is considered damning, in a court of law, and a jury of my peers, I need a better lawyer.
It would not be "damning" - as in you won't be convicted as a drug lord based on that alone - but search histories had been used, many times, in a court of law as evidence supporting intent, knowledge and other details of the crime. So if you are caught in a cocaine lab holding a bag of cocaine, then in the following process asking ChatGPT on how to make it would be used as part of the proof you were interested in making cocaine and eventually made it.
Re: When ChatGPT turns informant
#60Earlier quoted context omitted.
I don't understand how you don't understand. Trying to recreate someone's internal thoughts and attitudes from looking at their search history is a pale imitation of this. Just the thought experiment of a customs officer asking ChatGPT to summarise your political viewpoints was eye opening to me.
How so? You'd have a very, very good understanding of my political viewpoints from the log of my Google searches. I'm asking sincerely, not simply to push back on you.
Asking Google for details about January 6th is different than telling ChatGPT I think the election was stolen, and then arguing with it for hours about it.
It would be harder to frame it in front of a jury that what you typed wasn't an accurate representation of what you were thinking and that you were being duplicitous to ChatGPT.