Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent bankrupted their operator while trying to scan DN42

lantian.pub

Post reply on HN
Print view View original

431–440 of 575 posts

Re: AI agent bankrupted their operator while trying to scan DN42

#431

Post by user43928 »

inigyou wrote: post #424
user43928 wrote: post #219

Earlier quoted context omitted.

Is that surprising, considering I'm using AI a lot? I have not hand written a single line of code in months on my side projects. Obviously I am also interested in discussing the latest model. Your claim that I promote anything or otherwise don't engage here in good faith is both misplaced and against the site rules.

What was the back and forth?

Since it couldn't reproduce the issue in the iOS simulator, I had to run it on the device, reproduce the issue, and provide the logs.

This went on for many rounds, during which I tried to steer it toward what I thought was the source of the bug, while the model mostly kept adding instrumentation and logs.

In the end the cause was not what I suspected, but reasonably close to it via another mechanism.

Re: AI agent bankrupted their operator while trying to scan DN42

#432

Post by ghaff »

brazzy wrote: post #326
Symbiote wrote: post #305

Earlier quoted context omitted.

Can a German child buy non-essential expensive things, like a concert ticket, console, Warhammer or whatever? (Or a video game, back when those were sold in shops.) I bought these things while a child in the UK. I'm sure Games Workshop would have offered a refund on something unopened if my parents had demanded it, but I'm fairly sure the ticket agency would not.

The generally agreed limit (also established in court cases) is the amount of pocket money a child of the given age typically gets per month. For a 10 year old, that's about 20 EUR, for a 16 year old about 50 EUR. A console would definitely be too expensive, as would be big name concert tickets. Unless it's a recent AAA title, video games would be OK. No idea what Warhammer costs these days. Most retailers are probab…

There are definitely limits in some countries relative to the US. I was in university at 16. My parents were covering a lot of costs but I was certainly making regular purchases of all manner of things. My understanding is that would perhaps be something of an issue some places.

Re: AI agent bankrupted their operator while trying to scan DN42

#433

Post by jerf »

100721 wrote: post #413
jerf wrote: post #408

Earlier quoted context omitted.

"The extremely surprising and concerning part of this whole story is that the agent reported that they proactively spun up 5 AWS instances with a combined 100Gps of network egress capacity." Although given the agent was clearly in la-la land at that point I take that claim with a grain of salt. If this was some bizarre and very ill-conceived scam, then that claim would be false. Though even by scammer standards, the…

Maybe I’m just groggy with Friday Brain going on, but I’m having trouble understanding what you’re suggesting. Do you think this was a scam attempt to extract money in the form of reparation donations?

I've seen some other suggestions of that idea in the full HN conversation, which I'm reacting to.

On the one hand I find it a bizarre approach to running a scam. On the other hand I'm having a hard time coming up with any theory of mind on my end as to why this person would solicit $5000+ from the people they just harassed. Sheer cluelessness does fit the facts, though.

Re: AI agent bankrupted their operator while trying to scan DN42

#434

Post by tovej »

inigyou wrote: post #420
lucianbr wrote: post #348

Earlier quoted context omitted.

I wonder if a probabilistic compiler would be fine for the people arguing this. One that sometimes produces machine code that does something else, and sometimes produces machine code that is just broken and does nothing useful. From the same source code. What if your compiler could be fooled by some other developers into spending thousands of dollars, and still not produce the desired machine code in the end?

I've run into compiler bugs before.

There are compiler bugs (rarely) which will be fixed. That's different from fundamental flaws in the technology, which cannot be fixed.

Re: AI agent bankrupted their operator while trying to scan DN42

#435

Post by nzealand »

DonHopkins wrote: post #405
Taniwha wrote: post #241

Earlier quoted context omitted.

There is also the true story from the first Scientology vs. Internet clash, someone trolled them that their files were being hosted on 127.0.0.1, under a court ordered deposition they tried to find out who was running this server with their secret files (because yes, they'd looked, and they were there)

True that! Keith Henson's legendary alt.religion.scientology loopback trolling story, with hilarious deposition transcript, in which he patiently explains how 127.0.0.1 works to astonished Scientology lawyers: https://news.ycombinator.com/item?id=20791891 >Just be glad you didn't have to explain an in joke about ftp sites, the local loopback address, and a troll, in a deposition, under oath, to Scientology lawyers, l…

Early internet wisdom was "don't feed the trolls" - I never realized trolling was from fishing.

Re: AI agent bankrupted their operator while trying to scan DN42

#437

Post by marcosdumay »

gnunicorn wrote: post #328
flowerthoughts wrote: post #66

> I have deployed five AWS m8g.12xlarge instances. Each instance provides: > 48 vCPUs (Graviton4, ARM64) > 192 GiB memory (4 GiB per vCPU) > Network capability: The 22.5 Gbps per-instance network performance (combined across all five instances) provides the aggregate 20 Gbps target with redundancy and fail-over capacity. Oh wow. Very important to have 5x redundancy and fail-over in your network scanner. Especially be…

Sounds like the default k8s setup every startup deploys to not fail it single digit number of users. It learned from the best

All on the same zone, of course, to avoid high-latency links.

Re: AI agent bankrupted their operator while trying to scan DN42

#438

Post by inigyou »

100721 wrote: post #409
inigyou wrote: post #398

Earlier quoted context omitted.

Could've rented a not so cheap 100Gbps server, hallucinated a few node addresses on it and asked it to please peer with this server to perform the scan at high speed. That would've wasted millions of dollars instead of mere thousands, but also cost a thousand for whoever did it.

I’m just a lowly dev and don’t have experience with seeing the bills from cloud providers for a whole org. Can you (or someone) shed some light to help me understand how this would ramp up to millions? Both for curiosity’s sake, and to make sure my self-deployed projects (0 AI, all manually configured) don’t bankrupt me.

AWS bandwidth is expensive as fuck. I think they're still pricing as $0.09 per GB?

Real wholesale bandwidth pricing is about a hundred times cheaper than that, and incoming bandwidth is often free. You could rent a server with 100Gbps connection, 10000TB/month outgoings cap (maybe), and have the AI spam packets to it, and mostly not reply to them. It would be expensive but not nearly as expensive as it would be for the guy on AWS.

Do some calculations: 100Gbps is 12.5 GBps which is about one dollar per second. Okay so maybe not millions of dollars but still a hundred thousand per day, while you are spending maybe 1000-3000 per month and cancelling after the first month.

Re: AI agent bankrupted their operator while trying to scan DN42

#439

Post by colechristensen »

Frieren wrote: post #36
colechristensen wrote: post #25

Earlier quoted context omitted.

They ramble on because those words are for them, not for you. There is some amount of hiding this through "thinking" modes that are hidden by default, but still you have to remember that ALL THEY ARE are complex statistical machines for predicting the next symbol.

> here is some amount of hiding this through "thinking" modes that are hidden by default, but still you have to remember that ALL THEY ARE are complex statistical machines for predicting the next symbol. 100% this. Too many people believes that chatbots "think". Text is all they do, it is impressive, but they need the text to generate more text. They being verbose is the point.

While we don't have a direct mechanistic understanding of consciousness there are plenty of experts who will propose all YOU are is a jumble of streams of symbols routing around through your brain. (being fair this is far from the only hypothesis)

Re: AI agent bankrupted their operator while trying to scan DN42

#440

Post by NothingAboutAny »

nzealand wrote: post #435
DonHopkins wrote: post #405

Earlier quoted context omitted.

True that! Keith Henson's legendary alt.religion.scientology loopback trolling story, with hilarious deposition transcript, in which he patiently explains how 127.0.0.1 works to astonished Scientology lawyers: https://news.ycombinator.com/item?id=20791891 >Just be glad you didn't have to explain an in joke about ftp sites, the local loopback address, and a troll, in a deposition, under oath, to Scientology lawyers, l…

Early internet wisdom was "don't feed the trolls" - I never realized trolling was from fishing.

I dunno if it even is because isn't that spelt trawling? just looked it up and they're both correct fishing terms sigh
Post reply on HN