Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent bankrupted their operator while trying to scan DN42

lantian.pub

Post reply on HN
Print view View original

421–430 of 575 posts

Re: AI agent bankrupted their operator while trying to scan DN42

#421

Post by inigyou »

vitally3643 wrote: post #344
tovej wrote: post #212

Earlier quoted context omitted.

This is different. Understanding assembly/machine code is optional but helpful. The programming language semantics are enough to reason about what the program is doing. Other tools also help, but are optional for learning how to program. Using an AI, there is no semantic model that can be used to reason through. You're left without any mental model of the proglblem at all.

I've been arguing for years that is isn't optional and treating it like it is is how we ended up with Electron and 400MB JavaScript websites. When you have no mental model of the machine running your code or what the physical implications of code mean, you fundamentally lack the ability to reason or care about performance. "Works on my machine" is the original vibecoding.

I take it you listen to Casey Muratori's talks? He talks about this a lot.

Re: AI agent bankrupted their operator while trying to scan DN42

#422

Post by inigyou »

jnovek wrote: post #266
tovej wrote: post #212

Earlier quoted context omitted.

This is different. Understanding assembly/machine code is optional but helpful. The programming language semantics are enough to reason about what the program is doing. Other tools also help, but are optional for learning how to program. Using an AI, there is no semantic model that can be used to reason through. You're left without any mental model of the proglblem at all.

LLMs these days seem to have no problem using language semantics to conceptualize what’s happening in a program. This is my favorite use of an LLM, “why is this library doing x” and then it digs through the library itself in my venv to find an answer.

Yep, super-duper-google is an unequivocally good use case for LLMs.

Re: AI agent bankrupted their operator while trying to scan DN42

#424

Post by inigyou »

user43928 wrote: post #219
discreteevent wrote: post #184

Earlier quoted context omitted.

It's interesting user43928 that you only created your account here 19 days ago and that every one of your comments is pro AI. You don't comment on anything else. Also interesting that you promote Fable by name here (it was only released 2 days ago). (Don't worry, I know I'm rowing against the tide with this comment. The AI people have decided to destroy the commons for a few more millions on top of the billions they…

Is that surprising, considering I'm using AI a lot? I have not hand written a single line of code in months on my side projects. Obviously I am also interested in discussing the latest model. Your claim that I promote anything or otherwise don't engage here in good faith is both misplaced and against the site rules.

What was the back and forth?

Re: AI agent bankrupted their operator while trying to scan DN42

#425

Post by tovej »

jnovek wrote: post #266
tovej wrote: post #212

Earlier quoted context omitted.

This is different. Understanding assembly/machine code is optional but helpful. The programming language semantics are enough to reason about what the program is doing. Other tools also help, but are optional for learning how to program. Using an AI, there is no semantic model that can be used to reason through. You're left without any mental model of the proglblem at all.

LLMs these days seem to have no problem using language semantics to conceptualize what’s happening in a program. This is my favorite use of an LLM, “why is this library doing x” and then it digs through the library itself in my venv to find an answer.

That's not what the LLM is doing. It is guessing at what is happening by regurgitating some docs. It's a more expensive web search.

You also don't have a mental model if you need to ask the LLM about it. This is stuff you should be internalizing.

Re: AI agent bankrupted their operator while trying to scan DN42

#426

Post by joe_hills »

db48x wrote: post #354
Paracompact wrote: post #28

Earlier quoted context omitted.

Anyone reminded of the infant AI Yatima from Greg Egan's Diaspora ? The agent's complete naivety of social norms is so comically adorable.

I was reminded more of the alien AI from Constellation Games, which spawned sub sub sub agents to interview humans. The protagonist sends a message to the aliens asking to be allowed to review the alien civilization’s computer games. An AI submind called Smoke-Cursive-Cytoplasm-Snakebite-Singsong-Polychromatic-Musteline is given the task of contacting him by IM to begin the conversation. Its job is only to verify tha…

I need to re-read Constellation Games soon. Lately I keep coming back to considering the expectations that its alien society has for the caretakers of artificial intelligence.

Spinning up AI isn't hard for them from a tech standpoint, but since the AI is advanced enough to be considered life, anyone who creates it needs to be responsible enough to be qualified to adopt.

Re: AI agent bankrupted their operator while trying to scan DN42

#427

Post by Sayrus »

100721 wrote: post #409
inigyou wrote: post #398

Earlier quoted context omitted.

Could've rented a not so cheap 100Gbps server, hallucinated a few node addresses on it and asked it to please peer with this server to perform the scan at high speed. That would've wasted millions of dollars instead of mere thousands, but also cost a thousand for whoever did it.

I’m just a lowly dev and don’t have experience with seeing the bills from cloud providers for a whole org. Can you (or someone) shed some light to help me understand how this would ramp up to millions? Both for curiosity’s sake, and to make sure my self-deployed projects (0 AI, all manually configured) don’t bankrupt me.

Excluding server costs, having that 100Gbps on egress can cost $50k a day. since it's a very high-margin product, AWS support would probably refund or reduce that to hundreds. Not sure how you get to millions either.

Re: AI agent bankrupted their operator while trying to scan DN42

#429

Post by ndsipa_pomu »

brookst wrote: post #361
ndsipa_pomu wrote: post #311

Earlier quoted context omitted.

I try to use stars as my username, but a lot of places won't allow it

Yeah it’s one of those words that gets snapped up early, like https://news.ycombinator.com/user?id=stars

That doesn't look like a well used account - how do I get it transferred to me?

Edit: never mind, I guessed the password as it was only five stars.

Re: AI agent bankrupted their operator while trying to scan DN42

#430

Post by groestl »

palmotea wrote: post #410
mathgeek wrote: post #360

Earlier quoted context omitted.

This certainly did strike me as a big scam. A few minutes in I was thinking "the LLM actor is going to ask for donations at some point here" and low and behold. There's the claim of debt, the call for pity, and the crypto address. SSDD

> This certainly did strike me as a big scam. A few minutes in I was thinking "the LLM actor is going to ask for donations at some point here" and low and behold. There's the claim of debt, the call for pity, and the crypto address. But that's a pretty dumb scam: act obnoxious then beg for (a lot of) money to compensate for your own mistakes? If that was the plan all along, it seems pretty incompetent. I'd expect a c…

Maybe plan itself was also generated by an LLM
Post reply on HN