Scamlexity: When agentic AI browsers get scammed • Hacker News Board

Re: Scamlexity: When agentic AI browsers get scammed

#141

Post by tsimionescu » Mon, Aug 25, 2025, 6:04 PM UTC

ninetyninenine wrote: post #132

tsimionescu wrote: post #103

Earlier quoted context omitted.

First of all, medieval style punishments is not an acceptable answer. Scamming someone doesn't rise anywhere near the level of offense that should forfeit your human rights. The people who made this law and those who actually carry out the punishments are far more deserving of the death penalty than the vast majority of scammers. Now, even disregarding this obvious violation of human rights, from even a purely amoral…

>First of all, medieval style punishments is not an acceptable answer. Why? just because you put your foot down it's not acceptable? Think about it from another perspective. Think in terms of effectiveness rather than compassion. If compassion results in shit holes like SF, while strict punishment results in singapore. You can't argue with results. Like I get your argument. Everyone gets it. Solutions cannot however…

No, effectiveness is not an excuse for immoral, disproportionate punishments. And morality is not nearly as irrational or difficult as you make it out to be. The victims of a scammer are not nearly as badly hurt as the victims of a pedophile. And since both crimes are perpetrated by knowing adults, there's no "they couldn't help it" compassion for the victims (note that it's not illegal to have pedophilic tendencies - it's illegal to hurt children by acting on those tendencies). So, the moral calculus is simple: same internal culpability for the perpetrator, but different levels of damage to the victims results in different levels of moral culpability.

And even for such heinous crimes, the death penalty is not acceptable, nor is corporal punishment. There is still value in a human life beyond such crimes. In addition, there is always the problem of applying major punishments to people who are actually innocent - which is a far more common occurence than proponents of such punishments typically admit. How happy would you be to be killed because you got confused for a scammer?

Not to mention, the deterrence effect is vastly overstated - there is little evidence of a significant difference in rates of major crime depending on the level of punishment, beyond some relatively basic level. Actual success rates of enforcement are a much more powerful predictor of crime rates. You can have the worse possible punishments, but if almost no one gets convicted, criminals will keep doing it hoping they won't personally get caught.

Re: Scamlexity: When agentic AI browsers get scammed

#142

Post by red-iron-pine » Mon, Aug 25, 2025, 6:04 PM UTC

disgruntledphd2 wrote: post #85

mh- wrote: post #69

Earlier quoted context omitted.

Supposing I accept that's a likely outcome, it's exactly the same thing that would have happened if a typical human shopper searched for Vitamin D and picked the top result, right? The cynicism on these topics is getting exhausting.

> Supposing I accept that's a likely outcome, it's exactly the same thing that would have happened if a typical human shopper searched for Vitamin D and picked the top result, right? Yeah sure, but humans (normally) only fall for a particular scam once. Because LLMs have no memory, they can scale these scams much more effectively!

and don't forget the one-day-of-entire-nation-of-bolivia tier electricity consumption just to get those dubious scans done

Re: Scamlexity: When agentic AI browsers get scammed

#143

Post by danaris » Mon, Aug 25, 2025, 6:07 PM UTC

achierius wrote: post #101

danaris wrote: post #64

Earlier quoted context omitted.

I don't think we'll need to turn off AIs because I don't think anything we're doing today is actually at any real risk of leading to an AI that's conscious and has its own opinions and agendas. What we've got is a very interesting text predictor. ...But also, what, exactly, is your imagination telling you that a hypothetical AGI without any connection to the outside world can do if it gets mad at us? If it doesn't ha…

You're proposing something that doesn't exist in reality: an LLM widely deployed in a way that totally isolates it from the outside world. That's not actually how we do things, so I don't understand why you seem to expect the Anthropic researchers to use that as their starting point. If you were to try and argue that we should change over existing systems to look more like your idealized version, you would in fact pr…

...No, I'm proposing something that is, in fact, the default (or at least it was until relatively recently, with the "agentic" LLMs): an LLM whose method of interacting with the world is entirely through the chat prompts. Input is either chat prompts, the system prompt, or its training, which is done offline.

It is absolutely not the normal thing to give an LLM tools to control your smart home, your Amazon account, or your nuclear missile systems. (Not because LLMs are ready to turn into self-aware AIs that can take over our world. Because LLMs are dumb, and cannot possibly be made to understand what's actually a good, sane way to use these things.)

...Also, I don't in any way buy the argument in favor of breaking people's things and putting them in actual danger to show them they need to protect themselves better. That's how you become the villain of any number of sci-fi or fantasy stories. If Anthropic genuinely believes that giving LLMs these capabilities is dangerous, the responsible thing to do is not do that with their own, while loudly and firmly advising everyone else against it too.

Re: Scamlexity: When agentic AI browsers get scammed

#144

Post by ninetyninenine » Mon, Aug 25, 2025, 6:16 PM UTC

tsimionescu wrote: post #141

ninetyninenine wrote: post #132

Earlier quoted context omitted.

>First of all, medieval style punishments is not an acceptable answer. Why? just because you put your foot down it's not acceptable? Think about it from another perspective. Think in terms of effectiveness rather than compassion. If compassion results in shit holes like SF, while strict punishment results in singapore. You can't argue with results. Like I get your argument. Everyone gets it. Solutions cannot however…

No, effectiveness is not an excuse for immoral, disproportionate punishments. And morality is not nearly as irrational or difficult as you make it out to be. The victims of a scammer are not nearly as badly hurt as the victims of a pedophile. And since both crimes are perpetrated by knowing adults, there's no "they couldn't help it" compassion for the victims (note that it's not illegal to have pedophilic tendencies…

>No, effectiveness is not an excuse for immoral, disproportionate punishments. And morality is not nearly as irrational or difficult as you make it out to be. The victims of a scammer are not nearly as badly hurt as the victims of a pedophile.

Not true. You talk as if your views are universal fact. They are not. Effectiveness is THE only metric because what's the point if things are ineffective? Effectiveness is the driver while compassion is the cost. The more compassion the more ineffective things typically are. You need to balance the views but to balance the views you need to know the extremes. Why does Singapore work? Have you asked this question? Unlikely given your extreme view points.

At best you can just disagree with Singapore. But you can never really say that your view points are universal. Singapore chooses the make the trade off of compassion for effectiveness.

Secondly, I personally know scam victims who are worse off than pedophilia victims. Pedophilia can be a one time traumatizing act while a scam victim can lose a lifetime of work.

>Not to mention, the deterrence effect is vastly overstated - there is little evidence of a significant difference in rates of major crime depending on the level of punishment, beyond some relatively basic level. Actual success rates of enforcement are a much more powerful predictor of crime rates. You can have the worse possible punishments, but if almost no one gets convicted, criminals will keep doing it hoping they won't personally get caught.

Weed is rarely used in Singapore because of death penalty. It is highly effective. It is not overrated. There are many many example cases of it being highly effective. I believe about 15 people have been hanged.

Re: Scamlexity: When agentic AI browsers get scammed

#145

Post by npteljes » Mon, Aug 25, 2025, 6:41 PM UTC

ninetyninenine wrote: post #126

npteljes wrote: post #41

Earlier quoted context omitted.

It has not been done in Singapore. Scams are continuing issue for them, like the rest of the world. See this Singaporean police resource regarding scamming in 2024: https://www.police.gov.sg/-/media/Spf/Media-Room/Statistics/...

These numbers are global. You need to get numbers where the criminal is within the borders of Singapore.

Here's an article where they focus on just one type of scam, increasing in number, where migrant workers are exploited by loan sharks, on Singapore soil.

https://www.straitstimes.com/singapore/maids-lost-at-least-8...

All which is to say: Singapore hasn't "solved scams" as a problem. Furthermore, I also claim that scams are not a solvable problem.

Re: Scamlexity: When agentic AI browsers get scammed

#146

Post by red-iron-pine » Mon, Aug 25, 2025, 6:48 PM UTC

stronglikedan wrote: post #49

roxolotl wrote: post #10

Earlier quoted context omitted.

Yea I’ve been surprised about the risk conversations because they cannot do anything if you run them in a sandbox. But it seems like for many the assumed part was we’d hook LLMs into everything asap. It’s absolutely mind boggling.

> But it seems like for many the assumed part was we’d hook LLMs into everything asap The "many" are lazy, and agents require relatively low effort to implement for a big payoff, so naturally the many will flock to that.

> relatively low effort to implement

low effort? you're gonna use the same amount of power as Argentina uses in a day to give users easily-gamed, easily-compromised, poor-quality recommendations for stuff they could just as easily get at a local pharmacy?

Re: Scamlexity: When agentic AI browsers get scammed

#147

Post by npteljes » Mon, Aug 25, 2025, 6:51 PM UTC

ninetyninenine wrote: post #144

tsimionescu wrote: post #141

Earlier quoted context omitted.

No, effectiveness is not an excuse for immoral, disproportionate punishments. And morality is not nearly as irrational or difficult as you make it out to be. The victims of a scammer are not nearly as badly hurt as the victims of a pedophile. And since both crimes are perpetrated by knowing adults, there's no "they couldn't help it" compassion for the victims (note that it's not illegal to have pedophilic tendencies…

>No, effectiveness is not an excuse for immoral, disproportionate punishments. And morality is not nearly as irrational or difficult as you make it out to be. The victims of a scammer are not nearly as badly hurt as the victims of a pedophile. Not true. You talk as if your views are universal fact. They are not. Effectiveness is THE only metric because what's the point if things are ineffective? Effectiveness is the…

>what's the point if things are ineffective

To be human, for one.

Extreme example: all you need to do to end all scams (and other, human-caused ills in the world) to just kill all humans. No humans, no human-made horrors.

Or, in case we'd like living humans, they could be kept in a way where they can't interact with one another. Boom, human on human solved.

>Pedophilia can be a one time traumatizing act while a scam victim can lose a lifetime of work.

This is very offensive, and makes zero sense, not in itself, not in the context of your argument. Please do reconsider.

Re: Scamlexity: When agentic AI browsers get scammed

#148

Post by npteljes » Mon, Aug 25, 2025, 6:52 PM UTC

mmmllm wrote: post #22

ninetyninenine wrote: post #14

The problem is scams which is a solvable problem. Eliminate the scams and AI can’t be scammed. It’s been done. See Singapore. Basically if you’re a scammer and you’re caught, death penalty or public whipping. That eliminates scammers real quick.

Is Singapore flying to other countries to arrest them there too? /s

They cooperate with international and other local forces, as any police do. But, they haven't eliminated scams, as OP claimed, that's complete bull.

Re: Scamlexity: When agentic AI browsers get scammed

#149

Post by red-iron-pine » Mon, Aug 25, 2025, 7:01 PM UTC

average_r_user wrote: post #37

WJW wrote: post #8

Earlier quoted context omitted.

You don't have to guess about the military applications, it's all over the news. Even bog standard FPV drones that Ukraine is churning out at a rate of >100k/month have image recognition these days, so that if the video stream gets jammed they can finish off the mission autonomously. Even on a hobby level, ardupilot+openCV+cheap drone kit from amazon is a DIY project within the skill set of a significant part of the…

I find it both surprising and, fortunately, reassuring that despite how easy it is to buy inexpensive components on platforms like Amazon, Temu, or AliExpress, we have yet to see a wave of terrorist attacks in the busiest public spaces.

the average american cannot afford a surprise $500 expense. you think they can afford a drone swarm?

Re: Scamlexity: When agentic AI browsers get scammed

#150

Post by anal_reactor » Mon, Aug 25, 2025, 7:38 PM UTC

Paradigma11 wrote: post #36

anal_reactor wrote: post #32

Earlier quoted context omitted.

Imagine an agent being a roommate. They see that toilet paper is running out, they go to the supermarket, they buy more, they charge you money. All without you saying a word. Sure, it might not be your favorite brand, or the price might not be optimal, but realistically, the convenience of not having to think about buying toilet paper is definitely worth the price of having your roommate choose the details. After all…

Sure, but why would you use a nondeterministic LLM for that? LLMs can do things that we cant reasonably do with deterministic software. But everything that can be done deterministically should be done deterministically.

Because for vast majority of people, LLM is a superior interface over Raspberry Pi with crontab running a Python script with headless Firefox. Hard to believe, I know.