Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude for Chrome

anthropic.com

Post reply on HN
Print view View original

241–250 of 433 posts

Re: Claude for Chrome

#241

Post by whatevertrevor »

mark242 wrote: post #209
biggestfan wrote: post #8

According to their own blog post, even after mitigations, the model still has an 11% attack success rate. There's still no way I would feel comfortable giving this access to my main browser. I'm glad they're sticking to a very limited rollout for now. (Sidenote, why is this page so broken? Almost everything is hidden.)

11% success rate for what is effectively a spear-phishing attempt isn't that terrible and tbh it'll be easier to train Claude not to get tricked than it is to train eg my parents.

The kind of attack vector is irrelevant here, what's important is the attack surface. Not to mention this is a tool facilitating the attack, with little to no direct interaction with the user in some cases. Just because spear-phishing is old and boring doesn't mean it cannot have real consequences.

(Even if we agree with the premise that this is just "spear-phishing", which honestly a semantics argument that is irrelevant to the more pertinent question of how important it is to prevent this attack vector)

Re: Claude for Chrome

#242

Post by felarof »

parsabg wrote: post #126

I built a very similar extension [1] a couple of months ago that supports a wide range of models, including Claude, and enables them to take control of a user's browser using tools for mouse and keyboard actions, observation, etc. It's a fun little project to look at to understand how this type of thing works. It's clear to me that the tech just isn't there yet. The information density of a web page with standard rep…

Just dumping the raw DOM into the LLM context is brutal on token usage. We've seen pages that eat up 60-70k tokens when you include the full DOM plus screenshots, which basically maxes out your context window before you even start doing anything useful.

We've been working on this exact problem at https://github.com/browseros-ai/BrowserOS. Instead of throwing the entire DOM at the model, we hook into Chromium's rendering engine to extract a cleaner representation of what's actually on the page. Our browser agents work with this cleaned-up data, which makes the whole interaction much more efficient.

Re: Claude for Chrome

#243

Post by felarof »

adam_arthur wrote: post #207
parsabg wrote: post #126

I built a very similar extension [1] a couple of months ago that supports a wide range of models, including Claude, and enables them to take control of a user's browser using tools for mouse and keyboard actions, observation, etc. It's a fun little project to look at to understand how this type of thing works. It's clear to me that the tech just isn't there yet. The information density of a web page with standard rep…

The LLM should not be seeing the raw DOM in its context window, but a highly simplified and compact version of it. In general LLMs perform worse both when the context is larger and also when the context is less information dense. To achieve good performance, all input to the prompt must be made as compact and information dense as possible. I built a similar tool as well, but for automating generation of E2E browser t…

> The LLM should not be seeing the raw DOM in its context window, but a highly simplified and compact version of it.

Precisely! There is already something accessibility tree that Chromium rendering engine constructs which is a semantically meaningful version of the DOM.

This is what we use at BrowserOS.com

Re: Claude for Chrome

#244

Post by SchemaLoad »

chankstein38 wrote: post #141
mrs6969 wrote: post #127

I don’t know if this will make anything better. Internet is now filled with ai generated text, picture or videos. Like we havent had enough already, it is becaming more and more. We make ai agents to talk to each other. Someone will make ai to generate a form, many other will use ai to fill that form. Even worst, some people will fill millions of forms in matter of second. What is left is the empty feeling of having…

I was just talking about this same thing with someone. It's like emails. If, instead of writing an email, I gave AI some talking points and then told it to generate an email around that, then the person that I sent it to has AI summarize it.... What's the point of email? Why would we still use email at all? Just either send each other shorter messages through another platform or let LLMs do the entire communication f…

LLMs are basically only useful when they can utilise public information. They are great for answering questions because the answer to your question can be pulled from wikipedia and reddit. They are completely useless for writing emails because they don't have any more info than you give them. The only thing they can do is fluff them out with nothingness, when the receiver is than AI summerising to strip out.

Re: Claude for Chrome

#245

Post by mikepurvis »

threatofrain wrote: post #161
parsabg wrote: post #126

I built a very similar extension [1] a couple of months ago that supports a wide range of models, including Claude, and enables them to take control of a user's browser using tools for mouse and keyboard actions, observation, etc. It's a fun little project to look at to understand how this type of thing works. It's clear to me that the tech just isn't there yet. The information density of a web page with standard rep…

> Having LLMs book flights by interacting with the DOM is sort of like having them code a web app using assembly. The DOM is merely inexpensive, but obviously the answer can't be solely in the DOM but in the visual representation layer because that's the final presentation to the user's face. Also the DOM is already the subject of cat and mouse games, this will just add a new scale and urgency to the problem. Now peo…

I had the same thought that really an LLM should interact with a browser viewport and just leverage normal accessibility features like tabbing between form fields and links, etc.

Basically the LLM sees the viewport as a thumbnail image and goes “That looks like the central text, read that” and then some underlying skill implementation selects and returns the textual context from the viewport.

Re: Claude for Chrome

#246

Post by SchemaLoad »

carlosjobim wrote: post #165
chankstein38 wrote: post #141

Earlier quoted context omitted.

I was just talking about this same thing with someone. It's like emails. If, instead of writing an email, I gave AI some talking points and then told it to generate an email around that, then the person that I sent it to has AI summarize it.... What's the point of email? Why would we still use email at all? Just either send each other shorter messages through another platform or let LLMs do the entire communication f…

Communication by e-mail is for when you need a human decision. AI can't help with that. > Just either send each other shorter messages through another platform Why would you use another platform for sending shorter messages? E-Mail is instant and supported on all platforms.

Because email is spammed with marketing. If you send me an email at work there is a good chance I won't see it because I got 20 emails from every SaaS product news letter flooding the inbox. If you send me a message on slack there is a 100% chance I will see it.

Re: Claude for Chrome

#247

Post by dotproto »

parsabg wrote: post #126

I built a very similar extension [1] a couple of months ago that supports a wide range of models, including Claude, and enables them to take control of a user's browser using tools for mouse and keyboard actions, observation, etc. It's a fun little project to look at to understand how this type of thing works. It's clear to me that the tech just isn't there yet. The information density of a web page with standard rep…

Just took a quick glance at your extension and observed that it's currently using the "debugger" permission. What features necessitated using this API rather than leveraging content scripts and less invasive WebExtensions APIs?

Re: Claude for Chrome

#248

Post by jascha_eng »

asdff wrote: post #229
jascha_eng wrote: post #220

Earlier quoted context omitted.

Oh the tools are hand coded (or rather built with Claude Code) but the agent can call them to control the browser. Imagine a prompt like this: You are a research agent your goal is to figure out this companies tech stack: - Company Name Your available tools are: - navigate_to_url: use this to load a page e.g. use google or bing to search for the company site It will return the page content as well as a list of availa…

I still don't understand what the llm does. One could do this with a few lines of curl and a list of tools to query against.

The LLM understands arbitrary web pages and finds the correct links to click. Not for one specific page but for ANY company name that you give it.

It will always come back with a list of technologies used if available on the companies page. Regardless of how that page is structured. That level of generic understanding is simply not solveable with just some regex and curls.

Re: Claude for Chrome

#249

Post by felarof »

jascha_eng wrote: post #146
aliljet wrote: post #22

Having played a LOT with browser use, playwright, and puppeteer (all via MCP integrations and pythonic test cases), it's incredibly clear how quickly Claude (in particular) loses the thread as it starts to interact with the browser. There's a TON of visual and contextual information that just vanishes as you begin to do anything particularly complex. In my experience, repeatedly forcing new context windows between sc…

I have built a custom "deep research" internally that uses puppeteer to find business information, tech stack and other information about a company for our sales team. My experience was that giving the LLM a very limited set of tools and no screenshots worked pretty damn well. Tbf for my use case I don't need more interactivity than navigate_to_url and click_link. Each tool returning a text version of the page and th…

This is super cool!

If a "deep research" like agent is available directly in your browser, would that be useful?

We are building this at BrowserOS!

Re: Claude for Chrome

#250

Post by SchemaLoad »

ares623 wrote: post #138
mrs6969 wrote: post #127

I don’t know if this will make anything better. Internet is now filled with ai generated text, picture or videos. Like we havent had enough already, it is becaming more and more. We make ai agents to talk to each other. Someone will make ai to generate a form, many other will use ai to fill that form. Even worst, some people will fill millions of forms in matter of second. What is left is the empty feeling of having…

Some of us won’t. But a majority probably will. Even more important, the kids of today won’t care. Their internet will be fully slopped. And with outdoor places getting more and more rare/expensive, they’ll have no choice but to consume slop.

The only solution I see is taxes going to fund outdoor in person spaces. As a society we very easily can afford these spaces, it's just that the people who need them most are the ones least able to pay for things.

Banning social media for kids alongside funding free or subsidised in person environments will be a huge benefit to society.

Post reply on HN