Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude for Chrome

anthropic.com

Post reply on HN
Print view View original

401–410 of 433 posts

Re: Claude for Chrome

#401

Post by dfabulich »

lionkor wrote: post #342
dfabulich wrote: post #272

Claude for Chrome seems to be walking right into the "lethal trifecta." https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ "The lethal trifecta of capabilities is:" • Access to your private data —one of the most common purposes of tools in the first place! • Exposure to untrusted content —any mechanism by which text (or images) controlled by a malicious attacker could become available to your LLM • The abilit…

So far the accepted approach is to wrap all prompts in a security prompt that essentially says "please don't do anything bad". > Prompt guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code. https://news.ycombinator.com/item?id=41864014 > - Inclusion prompt: User's travel preferences and food choices - Exclusion prompt: Credit card details, passport number, S…

There are better approaches, where you have dual LLMs, a Privileged LLM (allowed to perform actions) and a Quarantined LLM (only allowed to produce structured data, which is assumed to be tainted), and a non-LLM Controller managing communication between the two.

See also CaMeL https://simonwillison.net/2025/Apr/11/camel/ which incorporates a type system to track tainted data from the Quarantined LLM, ensuring that the Privileged LLM can't even see tainted data until it's been reviewed by a human user. (But this can induce user fatigue as the user is forced to manually approve all the data that the Privileged LLM can access.)

Re: Claude for Chrome

#402

Post by kridsdale1 »

DrewADesign wrote: post #390
chamomeal wrote: post #385

Earlier quoted context omitted.

I remember when people figured out you could tell bing chat “don’t use emoji’s or I’ll die” and it would just go absolutely crazy. Feel like there was a useful lesson in that. In fact in my opinion, if you haven’t interacted with a batshit crazy, totally unhinged LLM, you probably don’t really get them. My dad is still surprised when an LLM gives him an answer that isn’t totally 100% correct. He only started using ch…

> “it sounds very confident and looks correct, so this thing must be an all-knowing oracle”. I think the majority of the population will respond similarly, and the consequences will either force us to make the “note: this might be full of shit” disclaimer much larger, or maybe include warnings in the outputs. It’s not that people don’t have critical thinking skills— we’ve just sold these things as magic answer machin…

The psychological bug that confidence exploits is ancient and genetically ingrained in us. It’s how we choose our leaders and assess skilled professionals.

It’s why the best advice for young people is “fake it until you make it”

Re: Claude for Chrome

#403

Post by kachapopopow »

Sitting in a corner I have a hat, won't say what kind of hat, but it has been gathering dust for years. They're making it really hard to not dust it off. I can't imagine how many malicious people will be exploting this.

Re: Claude for Chrome

#404

Post by slashdev »

dfabulich wrote: post #272

Claude for Chrome seems to be walking right into the "lethal trifecta." https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ "The lethal trifecta of capabilities is:" • Access to your private data —one of the most common purposes of tools in the first place! • Exposure to untrusted content —any mechanism by which text (or images) controlled by a malicious attacker could become available to your LLM • The abilit…

It’s going to be pretty easy to embed instructions to Claude in a malicious website telling it to submit sensitive things (and not report that is doing it.)

Then all you have to do is get Claude to visit it. I’m sure people will find hundreds of creative ways to achieve that.

Re: Claude for Chrome

#405

Post by bluebarbet »

DrewADesign wrote: post #390
chamomeal wrote: post #385

Earlier quoted context omitted.

I remember when people figured out you could tell bing chat “don’t use emoji’s or I’ll die” and it would just go absolutely crazy. Feel like there was a useful lesson in that. In fact in my opinion, if you haven’t interacted with a batshit crazy, totally unhinged LLM, you probably don’t really get them. My dad is still surprised when an LLM gives him an answer that isn’t totally 100% correct. He only started using ch…

> “it sounds very confident and looks correct, so this thing must be an all-knowing oracle”. I think the majority of the population will respond similarly, and the consequences will either force us to make the “note: this might be full of shit” disclaimer much larger, or maybe include warnings in the outputs. It’s not that people don’t have critical thinking skills— we’ve just sold these things as magic answer machin…

>It’s not that people don’t have critical thinking skills

It isn't? I agree that it's a fallacy to put this down to "people are dumb", but I still don't get it. These AI chatbots are statistical text generators. They generate text based on probability. It remains absolutely beyond me why someone would assume the output of a text generator to be the truth.

Re: Claude for Chrome

#406

Post by fwip »

asdff wrote: post #200
stusmall wrote: post #181

It's wild to see an AI company put out a press release that is basically "hey, you kids wanna see a loaded gun?" Normally all their public coms are so full of optimism and salesmanship around the potential. They are fully aware of how dangerous this is.

> "We conducted extensive adversarial prompt injection testing, evaluating 123 test cases representing 29 different attack scenarios. " Doesn't this seem like a remarkably small set of tests? And the fact that it took this testing to realize that prompt injection and giving the reigns to the AI agent is dangerous strikes me as strange that this wasn't anticipated while building the tool in the first place, before it…

And even after their mitigations on known attacks, the attacks were still successful 11% of the time!

To misquote the IRA - "[Scammers] only need to be lucky once, you need to be lucky every time." Even a 1% chance of getting pwned every time you get sent a malicious email is way too high. Plus the scammers aren't gonna rest on their laurels - they'll be iterating too.

Re: Claude for Chrome

#407

Post by dmix »

kachapopopow wrote: post #403

Sitting in a corner I have a hat, won't say what kind of hat, but it has been gathering dust for years. They're making it really hard to not dust it off. I can't imagine how many malicious people will be exploting this.

You can be a Cyber-AI Security Analyst for SV

Re: Claude for Chrome

#408

Post by DrewADesign »

bluebarbet wrote: post #405
DrewADesign wrote: post #390

Earlier quoted context omitted.

> “it sounds very confident and looks correct, so this thing must be an all-knowing oracle”. I think the majority of the population will respond similarly, and the consequences will either force us to make the “note: this might be full of shit” disclaimer much larger, or maybe include warnings in the outputs. It’s not that people don’t have critical thinking skills— we’ve just sold these things as magic answer machin…

>It’s not that people don’t have critical thinking skills It isn't? I agree that it's a fallacy to put this down to "people are dumb", but I still don't get it. These AI chatbots are statistical text generators . They generate text based on probability. It remains absolutely beyond me why someone would assume the output of a text generator to be the truth.

> These AI chatbots are statistical text generators

Be careful about trivializing the amount of background knowledge you need to parse that statement. To us that says a lot. To someone whose entire life has been spent getting really good at selling things, or growing vegetables, or fixing engines, or teaching history, that means nothing. There’s no analog in any of those fields that would give the nuance required to understand the implications of that. It’s not like they aren’t capable of understanding it; their only source of information about it is advertising, and most people just don’t have the itch to understand how tech stuff works under the hood— much like you’re probably not interested in what specific fertilizer was used to grow your vegetables, even though you’re ingesting them, often raw, and that fertilizer could be anything from a petrochemical to human shit— so they aren’t going to go looking on their own.

Re: Claude for Chrome

#409

Post by int0x29 »

lionkor wrote: post #396
JyB wrote: post #395

Earlier quoted context omitted.

No one think any form of "prompt engineering" "guardrails" are serious security measures right?

Check the links I posted :) Some do think that, yes.

We need regulation. The stubborn refusal to treat injection attacks seriously will cost a lot of people their data or worse.

Re: Claude for Chrome

#410

Post by brookst »

aydyn wrote: post #324
brookst wrote: post #303

Earlier quoted context omitted.

Is it possible to give your parents access to to your browser in a safe way?

Why do people keep going down this sophistry? Claude is a tool, a piece of technology that you use. Your parents are not. LLMs are not people.

If you think it's sophistry you're missing the point. Let's break it down:

1. Browsers are open ended tools

2. A knowledgeable user can accomplish all sorts of things with a browser

3. Most people can do very impactful things on browsers, like transferring money, buying expensive products, etc.

4. The problem of older people falling for scams and being tricked into taking self-harming actions in browsers is ancient; anyone who was family tech support in the 2000's remembers removing 15+ "helpful toolbars" and likely some scams/fraud that older relatives fell for

5. Claude is a tool that can use a browser

6. Claude is very likely susceptible to both old and new forms of scams / abuse, either the same ones that some people fall for or novel ones based on the tech

7. Anyone who is set up to take impactful actions in their browser (transferring money, buying expensive things) should already by vigilant about who they allow to use their browser with all of their personal context

8. It is reasonable to draw a parallel between tools like Claude and parents, in the sense that neither should be trusted with high-stakes browsing

9. It is also reasonable to take the same precautions -- allow them to use private browsing modes, make sure they don't have admin rights on your desktop, etc.

The fact that one "agent" is code and the other is human is totally immaterial. Allowing any agent to use your personal browsing context is dangerous and precautions should be taken. This shouldn't be surprising. It's certainly not new.

Post reply on HN