Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scamlexity: When agentic AI browsers get scammed

guard.io

Post reply on HN
Print view View original

81–90 of 198 posts

Re: Scamlexity: When agentic AI browsers get scammed

#81

Post by LtWorf »

benterix wrote: post #5

This should hit the headlines. I was always of the opinion that AI of all kinds is not a threat unless someone decides to connect it to an actuator so that it has direct and uncontrolled effect on the external world. And now it's happening en masse with agents, MCPs etc. I don't even mention things we don't know about (military and other classified projects).

Friend of mine told me they'll monitor the refuelling of planes with image recognition AI, just by seeing if the pipe is attached to the plane or not…

Re: Scamlexity: When agentic AI browsers get scammed

#82

Post by xenotux »

AlexandrB wrote: post #55
xenotux wrote: post #50

Earlier quoted context omitted.

> I don't understand why we would ever want an agent to buy stuff for us. Why not? Offload the entire task, not just one half of it. It's why many well-off people have accountants, assistants, or servants. And no one says "you know, I'm glad you prepared my taxes, but let me file the paperwork myself". I think what you're saying isn't that you like going through checkout flows, just that you don't trust the computer…

> It's why many well-off people have assistants or servants. AI agents have only one master - the AI vendor. They're not going to make decisions based on your best interests.

You can say that about 99% of the tech that people use today. Windows and MacOS don't serve you. Your browser doesn't serve you. Heck, Hacker News doesn't serve you - it serves a bunch of VCs!

But the reality is that most of the time, this is not an adversarial relationship; and when it is, we see it as an acceptable trade-off ("ok, so I get all this stuff for free, and in exchange, maybe I buy socks from a different company because of the ads").

I'm not saying it's an ideal state or that there are no hidden and more serious trade-offs, but I don't think that what you're saying is a particularly compelling point for the average user.

Re: Scamlexity: When agentic AI browsers get scammed

#83

Post by lubujackson »

jaimebuelta wrote: post #23

I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…

Exactly. If we really wanted AI to help us, we would find a way to fix the inscrutable problem of why we have to enter our ID number over a phone, then do it a second time when we connect to a human. No company in the world has solved this riddle.

Re: Scamlexity: When agentic AI browsers get scammed

#84

Post by nravic »

dumbfounder wrote: post #54
AlexandrB wrote: post #48

Earlier quoted context omitted.

Enjoy it while you can. Messing with which products get purchased by these agents is such a no-brainer revenue stream for AI companies.

Then I will use a different service. I think this will be harder to monopolize than search.

I think even easier in fact - what's happening behind the scenes w/ an LLM is far more opaque

Re: Scamlexity: When agentic AI browsers get scammed

#85

Post by disgruntledphd2 »

mh- wrote: post #69
everdrive wrote: post #39

Earlier quoted context omitted.

>Agent, I need some vitamin D, can you find me the best deal for some rated in the top 5? "I have picked the best reviewed vitamin D on Amazon." (and, it's a knockoff in the mixed inventory, and now you're getting lead-laced nothing)

Supposing I accept that's a likely outcome, it's exactly the same thing that would have happened if a typical human shopper searched for Vitamin D and picked the top result, right? The cynicism on these topics is getting exhausting.

> Supposing I accept that's a likely outcome, it's exactly the same thing that would have happened if a typical human shopper searched for Vitamin D and picked the top result, right?

Yeah sure, but humans (normally) only fall for a particular scam once. Because LLMs have no memory, they can scale these scams much more effectively!

Re: Scamlexity: When agentic AI browsers get scammed

#87

Post by nemomarx »

achierius wrote: post #58
padolsey wrote: post #19

Earlier quoted context omitted.

This is why I've found the safety research conducted by the likes of Anthropic and OAI to be so confusing. Like when they said that models are likely to blackmail developers in order to avoid being 'turned off' [1]. What an utterly and obviously contrived and inevitable derivation of narratives from humans (science fiction and others) in the corpus. Nothing surprising or interesting. However, their hypothesis is pres…

So you don't think that we'll need to turn off AIs? Regardless of where their impulse to avoid such comes from, the fact that they'll attempt to avoid that is important. I think you haven't thought about this enough. Attempting to reduce the issue to cyber security basics betrays a lack of depth in either understanding or imagination.

how is that a certain fact? why would an llm agent avoid being turned off?

if you're talking about a hypothetical different system just build it so they don't want to stay on. there's no reason to emulate that part

Re: Scamlexity: When agentic AI browsers get scammed

#88

Post by tsimionescu »

jkrom3 wrote: post #79
jaimebuelta wrote: post #23

I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…

One other ancillary benefit is no more “impulse” buying. Unless of course the AI gets incentivized to do it, it will then bubble that impulse buy up to the consumers UI.

I imagine the exact opposite is far more likely - there will be a button for "get your AI agent to consider us!" that will be even easier to just click, since you know it won't just lead to an immediate purchase - but they know very well it will lead to a purchase down the line.

Re: Scamlexity: When agentic AI browsers get scammed

#89

Post by [deleted] »

dumbfounder wrote: post #26
jaimebuelta wrote: post #23

I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…

Agent, I need some vitamin D, can you find me the best deal for some rated in the top 5? Agent deployed. Ok we found a bottle with a 30 day supply of Nature’s Own from a well respected merchant. It can be here in 2 days and it is $12. Should I buy? Yes. Or you could add some other parameters and tell it to buy now if under $15. Agent, I need a regular order for my groceries, but I also need to make a pumpkin pie so c…

[deleted]

Re: Scamlexity: When agentic AI browsers get scammed

#90

Post by tsimionescu »

layer8 wrote: post #42
jaimebuelta wrote: post #23

I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…

It’s what wealthy people use human assistants for. If AI could do it as reliably, people would use that.

Key being wealthy. The kind of wealth that has no idea how much a banana costs, and couldn't care less whether it's 10 dollars or 1.
Post reply on HN