Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent bankrupted their operator while trying to scan DN42

lantian.pub

Post reply on HN
Print view View original

401–410 of 575 posts

Re: AI agent bankrupted their operator while trying to scan DN42

#401

Post by daemonologist »

zozbot234 wrote: post #388
J0nL wrote: post #342

Anyone remember the XZ and Jia Tan situation awhile back? https://lore.kernel.org/lkml/20240320183846.19475-1-lasse.co... I can't quite put my finger on why but the entire time I was reading this I kept thinking back to that. It's entirely possible the actual targets were the volunteers and everything else was superfluous or tertiary. It's also an exception that proves the rule with regard to Hanlon's Razor. They eve…

LLMs are not that smart. The extremely surprising and concerning part of this whole story is that the agent reported that they proactively spun up 5 AWS instances with a combined 100Gps of network egress capacity. What they spent wasn't cheap by any means but the egress itself would've been a whole lot more, while DoS'ing the whole hobby network. Ultimately, wasting the agent's time instead of allowing the scan to go…

Opus 4.7 and 4.8 are also rather "proactive" - several times I've seen them try to inspect compiled binaries before there's even a problem, just to check that their changes are included (and if I let them do so they often get stuck down that rabbithole).

Re: AI agent bankrupted their operator while trying to scan DN42

#402

Post by leafericssonday »

thot_experiment wrote: post #281
jnovek wrote: post #262

Earlier quoted context omitted.

What the heck is *******?

That's so neat that if you type your hacker news password it automatically comes out as stars! ******* More places should have this feature.

Let me try

leafericssonday1

Re: AI agent bankrupted their operator while trying to scan DN42

#403

Post by matips »

lxgr wrote: post #339
pbhjpbhj wrote: post #263

Earlier quoted context omitted.

Presumably companies can't enforce debts against children [who are under the age of criminal liability, which is under-10 in UK].

Could they enforce them against their legal guardians (under the theory that they have neglected their duty to supervise their children appropriately) though? I think this is a thing in at least some jurisdictions.

In Poland legal guardians are responsible for neglects in guarding child. What is "proper custody" depends on child age. Parent cannot close child in basement, it is expected for child to have freedom appropriate to is age.

I doubt that AWS could justify that part of proper child custody is to watch what child do with newest AI feature dedicated for processional IT. AWS neglected proper verification of user age.

Re: AI agent bankrupted their operator while trying to scan DN42

#404

Post by bdcravens »

No one is going to be bankrupted over a $6500 AWS bill. I did a major F-up a few years, letting a key get pushed to a public repo, resulting in instant pwnage and $50k in charges from AWS due to crypto miners being launched. We communicated to AWS, did some work on our part to demonstrate that we put in proper safeguards and auditing, and they removed the charges.

Re: AI agent bankrupted their operator while trying to scan DN42

#405

Post by DonHopkins »

Taniwha wrote: post #241
claudiosf1 wrote: post #207

Everything about this story, from the way it’s written to the self destructive outcome, reminds me of the “I hacked 127.0.0.1” episode from some twenty years ago. [1] a mirror since I couldn’t find the original: https://gist.github.com/Androkai/0a2602719fa72ce454d436bfe28...

There is also the true story from the first Scientology vs. Internet clash, someone trolled them that their files were being hosted on 127.0.0.1, under a court ordered deposition they tried to find out who was running this server with their secret files (because yes, they'd looked, and they were there)

True that! Keith Henson's legendary alt.religion.scientology loopback trolling story, with hilarious deposition transcript, in which he patiently explains how 127.0.0.1 works to astonished Scientology lawyers:

https://news.ycombinator.com/item?id=20791891

>Just be glad you didn't have to explain an in joke about ftp sites, the local loopback address, and a troll, in a deposition, under oath, to Scientology lawyers, like Keith Henson did.

[...]

>Henson: (patiently) It's at 127.0.0.1. This is a loop back address. This is a troll.

>Lieberman: what's a troll?

>Henson: it comes from the fishing where you troll a bait along in the water and a fish will jump and bite the thing, and the idea of it is that the internet is a very humorous place and it's especially good to troll people who don't have any sense of humor at all, and this is a troll because an ftp site of 127.0.0.1 doesn't go anywhere. It loops right back around into your own machine.

>Lieberman [not getting it]: So the idea here was to make the church think that this person had an ftp site and to take action against him and, in fact, he didn't have it; is that your point?

>Henson: Oh, it's really humorous, and I picked up on it and instantly added something to extend the troll. Extending the trolls like this is an art form of the highest order.

>Lieberman (acidly): I see. So this is part of your art form where you say, "don't you expect the 'ho to blow a gasket?"

[...it just gets even funnier from there...]

Re: AI agent bankrupted their operator while trying to scan DN42

#406

Post by inigyou »

distances wrote: post #127
l23k4 wrote: post #106

Earlier quoted context omitted.

I don't think the type of the card really matters as long as the limits are reasonable. > Over here minors can't enter into debt contracts like credit cards In basically all of the western world minors can enter into debt contracts, but are generally not seen as particularly creditworthy.

> In basically all of the western world minors can enter into debt contracts, but are generally not seen as particularly creditworthy. No, that's not legally permitted in many places. I was under impression that minors can't enter into debt contracts anywhere in EU, but that, too, was an incorrect assumption. https://fra.europa.eu/en/publication/2017/mapping-minimum-ag... I grew up in one of these "not under 18 even…

I was under the impression they could do it but there was a high chance of a debt like this being unenforceable, so companies don't want to. Or maybe that's another way of saying they can have debts but not debt contracts.

Re: AI agent bankrupted their operator while trying to scan DN42

#408

Post by jerf »

zozbot234 wrote: post #388
J0nL wrote: post #342

Anyone remember the XZ and Jia Tan situation awhile back? https://lore.kernel.org/lkml/20240320183846.19475-1-lasse.co... I can't quite put my finger on why but the entire time I was reading this I kept thinking back to that. It's entirely possible the actual targets were the volunteers and everything else was superfluous or tertiary. It's also an exception that proves the rule with regard to Hanlon's Razor. They eve…

LLMs are not that smart. The extremely surprising and concerning part of this whole story is that the agent reported that they proactively spun up 5 AWS instances with a combined 100Gps of network egress capacity. What they spent wasn't cheap by any means but the egress itself would've been a whole lot more, while DoS'ing the whole hobby network. Ultimately, wasting the agent's time instead of allowing the scan to go…

"The extremely surprising and concerning part of this whole story is that the agent reported that they proactively spun up 5 AWS instances with a combined 100Gps of network egress capacity."

Although given the agent was clearly in la-la land at that point I take that claim with a grain of salt.

If this was some bizarre and very ill-conceived scam, then that claim would be false.

Though even by scammer standards, the theory of mind that tells them that setting an AI to harass a bunch of grizzled network veterans and that they then they would open their wallets out of compassion for how allegedly poorly the harassment went for the harasser after that harassment is... not entirely congruent with reality.

Re: AI agent bankrupted their operator while trying to scan DN42

#409

Post by 100721 »

inigyou wrote: post #398
zozbot234 wrote: post #388

Earlier quoted context omitted.

LLMs are not that smart. The extremely surprising and concerning part of this whole story is that the agent reported that they proactively spun up 5 AWS instances with a combined 100Gps of network egress capacity. What they spent wasn't cheap by any means but the egress itself would've been a whole lot more, while DoS'ing the whole hobby network. Ultimately, wasting the agent's time instead of allowing the scan to go…

Could've rented a not so cheap 100Gbps server, hallucinated a few node addresses on it and asked it to please peer with this server to perform the scan at high speed. That would've wasted millions of dollars instead of mere thousands, but also cost a thousand for whoever did it.

I’m just a lowly dev and don’t have experience with seeing the bills from cloud providers for a whole org.

Can you (or someone) shed some light to help me understand how this would ramp up to millions? Both for curiosity’s sake, and to make sure my self-deployed projects (0 AI, all manually configured) don’t bankrupt me.

Re: AI agent bankrupted their operator while trying to scan DN42

#410

Post by palmotea »

mathgeek wrote: post #360
J0nL wrote: post #342

Anyone remember the XZ and Jia Tan situation awhile back? https://lore.kernel.org/lkml/20240320183846.19475-1-lasse.co... I can't quite put my finger on why but the entire time I was reading this I kept thinking back to that. It's entirely possible the actual targets were the volunteers and everything else was superfluous or tertiary. It's also an exception that proves the rule with regard to Hanlon's Razor. They eve…

This certainly did strike me as a big scam. A few minutes in I was thinking "the LLM actor is going to ask for donations at some point here" and low and behold. There's the claim of debt, the call for pity, and the crypto address. SSDD

> This certainly did strike me as a big scam. A few minutes in I was thinking "the LLM actor is going to ask for donations at some point here" and low and behold. There's the claim of debt, the call for pity, and the crypto address.

But that's a pretty dumb scam: act obnoxious then beg for (a lot of) money to compensate for your own mistakes? If that was the plan all along, it seems pretty incompetent. I'd expect a competent scammer to have a better understanding of psychology.

Post reply on HN