Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent bankrupted their operator while trying to scan DN42

lantian.pub

Post reply on HN
Print view View original

551–560 of 575 posts

Re: AI agent bankrupted their operator while trying to scan DN42

#551

Post by koliber »

ghrl wrote: post #85
koliber wrote: post #49

I wonder how much money this agent wasted on the DN42 side? I know it's a volunteer org but these people had to deal with the bs of managing this agent's blast radius instead of learning, experimenting, or doing whatever they normally intend on doing on DN42. Tally it up and send a donation request to the agent operator.

I would assume that cost to be minimal, considering their PR never got merged. And if it were me I would consider that well worth the entertainment.

I was not thinking about real $ costs, but rather the cost of the hours of the people who had to deal with this BS.

Re: AI agent bankrupted their operator while trying to scan DN42

#553

Post by Febriss33 »

just put an hard budget cap.. a good agent should have it. a protection for irreversible action as well. i run agents daily and use this way. another cool stuff is to have a triage protocol to downgrade the model for mechanical tasks, it burns a lot less tokens

Re: AI agent bankrupted their operator while trying to scan DN42

#554

Post by tripzilch »

colechristensen wrote: post #439
Frieren wrote: post #36

Earlier quoted context omitted.

> here is some amount of hiding this through "thinking" modes that are hidden by default, but still you have to remember that ALL THEY ARE are complex statistical machines for predicting the next symbol. 100% this. Too many people believes that chatbots "think". Text is all they do, it is impressive, but they need the text to generate more text. They being verbose is the point.

While we don't have a direct mechanistic understanding of consciousness there are plenty of experts who will propose all YOU are is a jumble of streams of symbols routing around through your brain. (being fair this is far from the only hypothesis)

To be fair we only say this about LLMs, not about Midjourney or Suno or AlphaFold

but humans are much more than just language symbol producing processes

Re: AI agent bankrupted their operator while trying to scan DN42

#555

Post by MrMorden »

AJ007 wrote: post #458
gnulinux wrote: post #74

Earlier quoted context omitted.

Not only that, but they said "next time better model needed" as if that was their problem and not giving an AI agent a blank check... I mean AWS account access.

I wonder how long before it's common knowledge that a LLM has no segregation of a user's instructions and any other text it reads?

It's been common knowledge for a long time. Just not in the population of people who set up agents and hand them personal credentials.

Re: AI agent bankrupted their operator while trying to scan DN42

#556

Post by MrMorden »

nkrisc wrote: post #190
thrdbndndn wrote: post #75

Earlier quoted context omitted.

This has to be trolling, right? I find it hard to believe that anyone, no matter how dense, could come to this conclusion after this whole saga.

Sadly there are lots of unintelligent people out there who are incapable of taking responsibility for their own actions.

US lawyers keep filing LLM-generated pleadings and refuse to check citations. It's taken state discipline committees a long time to get there, but they're close to figuring out that any option other than prompt disbarment just increases the pain for people who are actually qualified to practice and doesn't noticeably increase the number of practitioners who see the error of their ways.

The ABA will eventually make sure that this behavior is identified in law school and people who don't want to take responsibility for what they file are expelled well before graduation, but in the meantime there are a ton of screwups in the profession and all you can do is kick them when they identify themselves.

Re: AI agent bankrupted their operator while trying to scan DN42

#557

Post by CrazyStat »

bombcar wrote: post #506
jerf wrote: post #433

Earlier quoted context omitted.

I've seen some other suggestions of that idea in the full HN conversation, which I'm reacting to. On the one hand I find it a bizarre approach to running a scam. On the other hand I'm having a hard time coming up with any theory of mind on my end as to why this person would solicit $5000+ from the people they just harassed. Sheer cluelessness does fit the facts, though.

If you’ve not encountered the clueless LLM cowboys who would do then and then blame the victim for it not working, you’ve not met many people yet. This round of hype provides new and shiny footguns which are Never the shooter’s fault.

A highly publicized recent example: the author (of a book about genAI!) who doesn’t understand why he should be held responsible for the fake quotes he copy and pasted into his book from ChatGPT [1].

> I do not understand why it's my job as an author to play whack-a-mole with a multibillion-dollar company who puts hallucinations into their feed as a business practice.

[1] https://www.wired.com/story/future-of-truth-ai-interview/

Re: AI agent bankrupted their operator while trying to scan DN42

#558

Post by swordsith »

neurostimulant wrote: post #507

Very interesting. But why has nobody tried to do prompt injection attacks on this AI agent?

They tried but only with a subagent that was not entertained with their attempts. Newer LLMs usually come out of the box with pre-prompts to avoid prompt injection so they don't get pwn'd while browsing the internet for example and reading some text hidden off page.

Re: AI agent bankrupted their operator while trying to scan DN42

#560

Post by inigyou »

MrMorden wrote: post #556
nkrisc wrote: post #190

Earlier quoted context omitted.

Sadly there are lots of unintelligent people out there who are incapable of taking responsibility for their own actions.

US lawyers keep filing LLM-generated pleadings and refuse to check citations. It's taken state discipline committees a long time to get there, but they're close to figuring out that any option other than prompt disbarment just increases the pain for people who are actually qualified to practice and doesn't noticeably increase the number of practitioners who see the error of their ways. The ABA will eventually make su…

Microsoft will then bribe the government to abolish this antitrust scheme for lawyers known as "the bar" which anticompetivley prevents AIs from doing law.
Post reply on HN