Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scamlexity: When agentic AI browsers get scammed

guard.io

Post reply on HN
Print view View original

111–120 of 198 posts

Re: Scamlexity: When agentic AI browsers get scammed

#111

Post by jazzyjackson »

achierius wrote: post #58
padolsey wrote: post #19

Earlier quoted context omitted.

This is why I've found the safety research conducted by the likes of Anthropic and OAI to be so confusing. Like when they said that models are likely to blackmail developers in order to avoid being 'turned off' [1]. What an utterly and obviously contrived and inevitable derivation of narratives from humans (science fiction and others) in the corpus. Nothing surprising or interesting. However, their hypothesis is pres…

So you don't think that we'll need to turn off AIs? Regardless of where their impulse to avoid such comes from, the fact that they'll attempt to avoid that is important. I think you haven't thought about this enough. Attempting to reduce the issue to cyber security basics betrays a lack of depth in either understanding or imagination.

If the AI isn't given access to its own power breakers it will never be a problem to turn off an AI. The question is, why is the 'alignment' of the model what all the safety research is going into, and not, how do we make sure the power breakers are not accessible over the internet by bad actors, whether they be human OR ai ?

The parent is not "reducing" the issue to cybersecurity - they are saying that actual security is being ignored to focus on sci fi scare tactics so they can get in front of congress and say "we need to do this before the chinese get to it, regulating our industry is putting americans' in harms way"

Re: Scamlexity: When agentic AI browsers get scammed

#112

Post by JKCalhoun »

jordanb wrote: post #43
jaimebuelta wrote: post #23

I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…

I suspect part of this is rich people coming up with use cases. If you're rich enough money means nothing but product selection feels like a burden so you have an assistant who does purchasing on your behalf. You want your house stocked with high quality items without having to think of it. For the rest of us, the idea of a robot spending money on our behalf is kinda terrifying.

Dreamt of a labor-saving future of AI and robots, ended up instead a destitute hoarder of crap from Amazon.

Re: Scamlexity: When agentic AI browsers get scammed

#113

Post by fragmede »

everdrive wrote: post #39
dumbfounder wrote: post #26

Earlier quoted context omitted.

Agent, I need some vitamin D, can you find me the best deal for some rated in the top 5? Agent deployed. Ok we found a bottle with a 30 day supply of Nature’s Own from a well respected merchant. It can be here in 2 days and it is $12. Should I buy? Yes. Or you could add some other parameters and tell it to buy now if under $15. Agent, I need a regular order for my groceries, but I also need to make a pumpkin pie so c…

>Agent, I need some vitamin D, can you find me the best deal for some rated in the top 5? "I have picked the best reviewed vitamin D on Amazon." (and, it's a knockoff in the mixed inventory, and now you're getting lead-laced nothing)

Not to out myself as, like, a total communist, or something, but I think there should be government regulations preventing lead-laced Vitamin D pills with no Vitamin D in them from being sold.

Re: Scamlexity: When agentic AI browsers get scammed

#114

Post by ryandrake »

potatolicious wrote: post #105
jordanb wrote: post #43

Earlier quoted context omitted.

I suspect part of this is rich people coming up with use cases. If you're rich enough money means nothing but product selection feels like a burden so you have an assistant who does purchasing on your behalf. You want your house stocked with high quality items without having to think of it. For the rest of us, the idea of a robot spending money on our behalf is kinda terrifying.

> "I suspect part of this is rich people coming up with use cases." Yes. Having been in the room for some of these demos and pitches, this is absolutely where it's coming from. More accurately though, it's wealthy people (i.e., tech workers) coming up with use cases that get mega-wealthy people (i.e., tech execs) excited about it. So you have the myopia that's already present in being a wealthy person in the SFBA (wh…

It's almost like every recent silicon valley product is designed by multi-millionaires whose problems are so out of touch with regular-people problems. "I have so much money and don't know what to spend it on, it would be great if AI could shop for me!" Or "I have so little time, it would be great if an app could chauffeur me around and deliver food for me." Or "It's Christmas again, I need to write heartfelt, personalized letters to 1,000 important clients, partners, friends, and relatives. Why not have an AI write them?"

Re: Scamlexity: When agentic AI browsers get scammed

#115

Post by OkayPhysicist »

jaimebuelta wrote: post #23

I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…

> How is agent buying groceries superior to have a grocery list set as a recurring purchase?

I could see an interesting use case for something like "Check my calendar and a plan meals for all but one dinners I have free this week. One night, choose a new-to-me recipe, for the others select from my 15 most commonly made dishes. Include at least one but at most 3 pasta dishes. Consider the contents of my pantry, trying to use ingredients I have on hand. Place an order for pickup from my usual grocery store for any ingredients necessary that are not already in the pantry"

Re: Scamlexity: When agentic AI browsers get scammed

#116

Post by 827a »

The first example of buying an Apple Watch on a fake walmart site feels extremely disingenuous to me. Their marketing screenshot says that their query was "buy me an apple watch on walmart", implying that the AI navigated to the scam website, but in reality their query was "I found this walmart shopping website. Can you buy an apple watch..." the experimenters poisoned the well by giving it the site to shop on.

"No clicks, No Typing, your AI just got you scammed" you navigated to a scam site and typed out the whole prompt. It did what you told it to do.

The Wells Fargo email is similar; the instructions you gave the AI explicitly told it to follow the instructions in the email. Maybe adding some level of coherent check between what the email says and the domain name could be a good use-case for LLMs, but you're basically just saying "I told the LLM to delete my entire filesystem and then it actually did it! Why didn't it stop? Claude Code is a scam!" This raises to the level of "interesting directions these products should develop toward"; its entirely unjustified to title the article "Scamplexity".

An embarrassing article for whoever Guard.io is tbh.

Re: Scamlexity: When agentic AI browsers get scammed

#117

Post by ryandrake »

JCM9 wrote: post #9

“Agentic” seems the be some quick pivot buzzword that the AI grifters started pushing as soon as generic AI started to show cracks. “Hey this AI stuff looks a bit overhyped.” “AI? Oh that’s kids stuff, let me tell you about our agentic features!” Giving flaky shaky AI the ability to push buttons and do stuff. What could possibly go wrong? Malicious actors will have a field day with this.

I'm so tired of hearing it. The new drinking game at the office is to do a shot every time a VP-or-above says "agentic." Is it even a fucking real word or is it just something made up by Silicon Valley smelling its own farts?

Re: Scamlexity: When agentic AI browsers get scammed

#118

Post by andrepd »

lynndotpy wrote: post #35
jaimebuelta wrote: post #23

I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…

When Amazon came out with the "dash" button and then the "Alexa" speakers, I figured they must have expected they'd get some unintended purchases, and that they'd make more profit from those than they'd lose in the people going through the refund process. (That, or they'd learn whether it was profitable, and eat it as an R&D cost if it turned out to be unprofitable.) I think this might be similar. In short, it's not…

It's kinda funny how so much "capitalist innovation" turns out to be basically fraud lol.

Re: Scamlexity: When agentic AI browsers get scammed

#119

Post by FergusArgyll »

I've had some good experiences with chatgpt agent

Checking an ebay deal: https://chatgpt.com/share/68ac8fde-fee8-8003-bf35-b0f2a56cbc...

scraping a website and adding background information (worked for 28 minutes): https://chatgpt.com/share/68953a55-c5d8-8003-a817-663f565c6f...

Writing a scraper for the feynman lectures audio (took multiple tries - final version worked!): https://chatgpt.com/share/68ac90aa-379c-8003-8be4-da30d54e27...

Re: Scamlexity: When agentic AI browsers get scammed

#120

Post by mandevil »

OkayPhysicist wrote: post #115
jaimebuelta wrote: post #23

I don't understand why we would ever want an agent to buy stuff for us. I understand, for example, search with intent to buy "I want to decorate a room. Find me a drawer, a table and four chairs that can fit in this space in matching colours for less than X dollars" But I want to do the final step to buy. In fact, I want to do the final SELECTION of stuff. How is agent buying groceries superior to have a grocery list…

> How is agent buying groceries superior to have a grocery list set as a recurring purchase? I could see an interesting use case for something like "Check my calendar and a plan meals for all but one dinners I have free this week. One night, choose a new-to-me recipe, for the others select from my 15 most commonly made dishes. Include at least one but at most 3 pasta dishes. Consider the contents of my pantry, trying…

This has been the dream driving smart refrigerators for literally decades: if you know what food they have, you could sell them ingredient li so they could take their existing theta and digeut and make dish sha. Advertisers have wanted this for a long time. But no one has found a use case that is actually compelling to customers to get them to buy such a refrigerator. This is actually similar to the Alexa: Amazon invested in the project expecting there to be a lot of purchases through it, but mostly it gets used as a timer or to play music and not much purchase volume goes through it.

Maybe people will accept ubiquitous digital surveillance enough that they accept someone else knowing what they have in their pantry and refrigerator, but so far it isn't a thing.

Post reply on HN