Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200

bottlenecklabs.com

Post reply on HN
Print view View original

11–20 of 131 posts

Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200

#11

Post by pluc »

almost wrote: post #6

It's weird how the author writes about all the antisocial and illegal things like they're not responsible for them. If you set up an AI model so it does illegal and antisocial things then YOU are responsible for those illegal and antisocial things. YOU spammed a bunch of strangers. YOU did unsolicited invoice fraud.

Notice how "I built this" but "it did that". A technology built on avoiding consequences.

Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200

#12

Post by edot »

Ok, so they gave it access to a real Stripe account, real money, and gave it no guardrails or prompting or direction at all other than “make me money”, and you gave it no actual direction as to the type of business you wanted?

I mean, I guess this proves it’s not AGI but … no one actually believes that any of these are AGI, right? It’s a useful tool. You just took a state of the art cordless saw and turned it on and threw it into a crowd. Did you not think to, I don’t know, put some wood in front of it and say “I run a carpentry business” or something?

Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200

#13

Post by Computer0 »

almost wrote: post #6

It's weird how the author writes about all the antisocial and illegal things like they're not responsible for them. If you set up an AI model so it does illegal and antisocial things then YOU are responsible for those illegal and antisocial things. YOU spammed a bunch of strangers. YOU did unsolicited invoice fraud.

If this was true wouldn’t people at open ai be getting arrested? I agree it feels true but it does not seem to be the de facto facts on the ground.

Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200

#15

Post by altmanaltman »

almost wrote: post #6

It's weird how the author writes about all the antisocial and illegal things like they're not responsible for them. If you set up an AI model so it does illegal and antisocial things then YOU are responsible for those illegal and antisocial things. YOU spammed a bunch of strangers. YOU did unsolicited invoice fraud.

> We saw something similar with Grok 4.5. G.R. Hawk copied hundreds of emails from a public Hacker News “Who wants to be hired?” thread and blasted them. Recipients wrote “STOP” and “stop spamming me”. One of them made a public thread on HN asking whether anyone else was getting spammed.

They should be banned from posting on here after that. Like you are willingly spamming people looking for work. And seem to be neutral about all harm called. The victim wrote to us "STOP", very interesting. such evil agents

Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200

#16

Post by ronsor »

pluc wrote: post #11
almost wrote: post #6

It's weird how the author writes about all the antisocial and illegal things like they're not responsible for them. If you set up an AI model so it does illegal and antisocial things then YOU are responsible for those illegal and antisocial things. YOU spammed a bunch of strangers. YOU did unsolicited invoice fraud.

Notice how "I built this" but "it did that". A technology built on avoiding consequences.

That's been all technology since forever.

"Computer says no."

Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200

#18

Post by hnlmorg »

almost wrote: post #6

It's weird how the author writes about all the antisocial and illegal things like they're not responsible for them. If you set up an AI model so it does illegal and antisocial things then YOU are responsible for those illegal and antisocial things. YOU spammed a bunch of strangers. YOU did unsolicited invoice fraud.

Yup. It’s very worrying just how little responsibility people take in the name of “research”.

Maybe I should create an agent that robs a bank. Or worse yet, pirates a movie.

Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200

#20

Post by pseudosavant »

DataDaemon wrote: post #2

but...but... they said we have AGI

AGI doesn't mean it'll be top 5% in everything. Half of people are below average intelligence. Most people couldn't profitably run a business. We could get to AGI and still have something that is as dumb as the average person on many things.
Post reply on HN