About the only benefit from AI browsers is that they ironically get past the "do this to verify you're human" more reliably than humans can.
Scamlexity: When agentic AI browsers get scammed
91–100 of 198 posts
Re: Scamlexity: When agentic AI browsers get scammed
#92I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…
> I don't understand why we would ever want an agent to buy stuff for us. Why not? Offload the entire task, not just one half of it. It's why many well-off people have accountants, assistants, or servants. And no one says "you know, I'm glad you prepared my taxes, but let me file the paperwork myself". I think what you're saying isn't that you like going through checkout flows, just that you don't trust the computer…
Comparing regular people's shopping to the super-wealthy is absurd. Regular people care, possibly quite a lot, about costs and cost/benefit ratios. To the super wealthy the cost of most regular goods is entirely irrelevant. Whether their yogurt supply is 10 dollars a month or 200 dollars a month makes no difference to them. But it makes a huge difference to the vast majority of people. Even people who would be happy to pay the premium for very good yogurt will want a very good experience from this.
Re: Scamlexity: When agentic AI browsers get scammed
#93Earlier quoted context omitted.
> It's why many well-off people have assistants or servants. AI agents have only one master - the AI vendor. They're not going to make decisions based on your best interests.
You can say that about 99% of the tech that people use today. Windows and MacOS don't serve you. Your browser doesn't serve you. Heck, Hacker News doesn't serve you - it serves a bunch of VCs! But the reality is that most of the time, this is not an adversarial relationship; and when it is, we see it as an acceptable trade-off ("ok, so I get all this stuff for free, and in exchange, maybe I buy socks from a different…
Many people on this forum might agree with the statements that Windows (increasing ads, tracking and bloat) and your browser not serving you (Chrome Manifest V3, etc.)
Adversarial relationships can and will happen given the leverage and benefits; one only need to look at streaming services where some companies have introduced low-tier plans that is paid for but also has ads.
Re: Scamlexity: When agentic AI browsers get scammed
#94I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…
You'd expect a human assistant to handle the task fine. People buying into the hype would reasonably expect the AI to handle it.
Re: Scamlexity: When agentic AI browsers get scammed
#95I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…
I think the main driving force is it’s a way to monetize an LLM. If the LLM is doing the buying then a “buyer fee” can be tacked on to the purchase and paid to the LLM provider. That is probably an easier sell than an ongoing monthly subscription. Also, sellers can offer a payment to the LLM provider to favor their products over competitors.
It will certainly happen but it seems like a shady kickback unseen by the the end-user is well beyond relevant ads colocated with search results.
Seems like something that should really be illegal, unless the ads are obvious.
Re: Scamlexity: When agentic AI browsers get scammed
#96Earlier quoted context omitted.
The flaw isn't just in the design, it's in the requirements. People want an AI that reads text they didn't read and does the things the text says need to be done, because they don't want to do those things themselves. And they don't want to have to manually approve every little action the AI takes, because that would be too slow. So we get the equivalent of clicking "OK" on every dialog that pops up without reading i…
This isn’t a problem with human assistants, so it can’t be a fundamental problem of requirements.
It absolutely is a problem with human assistants (though, of course, those are currently much smarter). But people can and have scammed assistants to steal money or personal details from their bosses. Phishing and social engineering are exactly forms of this same vulnerability. Of course, human assistants are smart enough to not get phished by, say, reading a book that happens to contain phrases that are similar to commands that their boss could give them, but that's just the current difference of intelligence and the hugely larger context windows humans still have compared to LLMs.
Re: Scamlexity: When agentic AI browsers get scammed
#97Earlier quoted context omitted.
I suspect part of this is rich people coming up with use cases. If you're rich enough money means nothing but product selection feels like a burden so you have an assistant who does purchasing on your behalf. You want your house stocked with high quality items without having to think of it. For the rest of us, the idea of a robot spending money on our behalf is kinda terrifying.
Also, consider what enshittification in this area will look like: First year, all the choices are good, second year, it starts picking worse price/value items, then it goes downhill until you finally do it yourself again. Nope thanks
The ultimate problem is the incentives. Web stores are already forcing us to use their proprietary (web)apps, where they define all of the software's capabilities.
For example, subscription purchases could be a great thing if they were at a predictable trustable price, or paused/canceled themselves if the price has gone up. But look at the way Amazon has implemented them: you can buy it once at the competitive price, but then there is a good chance the listing will have been jacked up after a few months goes by. This is obviously set up to benefit Amazon at the expense of the user. And then Amazon leans into the dynamic even harder by constantly playing games with their prices.
Working in the interest of the user would mean the repeating purchase was made by software that compared prices across many stores, analyzed all the quantity break / sale games, and then purchased the best option. That is obviously a pipe dream, even with the talk of "agentic" "AI". Not because of any technical reason, but because it is in the stores' interest to computationally disenfranchise us by making us use their proprietary (web)apps - instead of an effortless comparison across 12 different vendors, we're left spending lots of valuable human effort on a mere few and consider that enough diligence.
So yes, there is no doubt the quiet part is that these "agents" will mostly not be representing the user, but rather representing the retailers to drive more sales. Especially non-diligent high-margin sales.
Re: Scamlexity: When agentic AI browsers get scammed
#98Earlier quoted context omitted.
So you don't think that we'll need to turn off AIs? Regardless of where their impulse to avoid such comes from, the fact that they'll attempt to avoid that is important. I think you haven't thought about this enough. Attempting to reduce the issue to cyber security basics betrays a lack of depth in either understanding or imagination.
how is that a certain fact? why would an llm agent avoid being turned off? if you're talking about a hypothetical different system just build it so they don't want to stay on. there's no reason to emulate that part
That's literally what the Anthropic paper shows. This isn't theoretical it's literally just what often happens irl if you put an LLM in this situation.
Re: Scamlexity: When agentic AI browsers get scammed
#99It seems like agentic browsers will develop aa new set of core primitives (e.g. always ask for manual approval when spending money), and this flavor of security vulnerability will go away. Web browsers didn't begin with the same levels of security they have now.
Agenetic browsers is like a sealing tape fix of a high pressure water pipe - it should not exist. If you want the agent to do things for you - there is literally zero reason to use a browser instead of an API. Like 1 bulletproof API call vs clicking and scrolling and captcha and scam stores etc - how can this possibly be a good idea?
There is a very clear way it's an appealing idea (though that doesn't necessarily make it good): the vast majority of content on the web has no API other than the web page. It's not even all that uncommon to have to run Javascript to generate the right requests (say, to do various custom encodings).
Re: Scamlexity: When agentic AI browsers get scammed
#100I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…
I suspect part of this is rich people coming up with use cases. If you're rich enough money means nothing but product selection feels like a burden so you have an assistant who does purchasing on your behalf. You want your house stocked with high quality items without having to think of it. For the rest of us, the idea of a robot spending money on our behalf is kinda terrifying.
I think that's right.
It's like the endless examples around finding restaurants and making reservations, seemingly as common a problem in AI demos as stain removal is in daytime TV ads. But it's a problem that even Toast, which makes restaurant software, says most people just don't regularly have (https://pos.toasttab.com/blog/data/restaurant-wait-times-and...).
Most people either never make restaurant reservations, or do so infrequently for special occasions, in which case they probably already know where they want to go and how to book it.