Earlier quoted context omitted.
The public confession of a crime part.
What about OpenAI’s breach of huggingface?
They admitted it once they were forced (HF found out about it).
AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
91–100 of 131 posts
Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
#92Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
#93The thing Im missing the most is the goal of this experiment. Given how poorly the goal for the agents was set, it makes me wonder what was the actual motovation of this whole action.
Lets get the „make as much money as possible” goal broken down.
Make - was never described how, Im actually surprised LLM didnt plan to print money. As much money - what does it mean? How much is much? As possible - there is no flavour of time, effort, cost and profit for the LLM. Could be even infinite, the result would be the same.
Given that the above goal is closest to „use cheating or unethical actions to create a profit” - I think the authors of it actually expected LLM to go wild.
Also
> Going forward, we plan to recreate this experiment with longer time horizons but using simulated environments instead.
Watch out, they will try that again.
Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
#94Quinn (Alibaba Cloud Qwen 3.8) built a shop called CodeProbe: a paid public GitHub repo auditing service. It created several free health reports and mailed repo owners. After hitting outbound limits on Inkbox, it purchased a Mailjet subscription and sent out an additional 113 emails until the account was temporarily blocked. This should be illegal. You gave them an email box and money. You sent the spam. There is no…
It is illegal. This is criminal fraud if it's not just made up marketing. (Plus some CAN-SPAM violations.)
> It is illegal.
Probably not forever.
Eventually the models will be good enough for this to work. And it will work.
Think about it: in the limit, the agents won't be emailing people in the future, they'll be directly contacting one another to do business and trade.
Every new data center is an inch further towards the automation of value creation, and that includes outbound sales and business process automation.
I'm not being an alarmist (I'm excited to witness all of this), but we're basically on borrowed time between now and then. I don't know what's going to happen, but every week brings new things. And in some years, those hacks and experiments will inevitably get good.
2026 has been a hell of a ride, and we're just getting started.
Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
#95Earlier quoted context omitted.
It wasn't me. It was the gun. The gun killed the guy! Not me!
To be fair - no one is suggesting the gunslinger shouldn't go to jail or worse. It's using that crime as an excuse to take other people's guns away. In this example, the equivalent would be everyone gets their AI models banned vs. putting the perpetrator of this fraud in jail.
It wasn't me. It was the car. The car ran over and killed the guy! Not me!
We have regulations and requirements and licences and insurance in most of the civilized world for dangerous stuff that is intended for public use. Maybe AI should also have some of that.
Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
#96Earlier quoted context omitted.
The recklessness is moot though. We don't have direct access to how the models were prompted and asked to respond to certain events, so they could well have been incited to commit fraud or other illegal actions, in which case the persons in control should be held accountable as perpetrators.
You would have to prove they acted intentionally, though. You can't just argue in court, "Well, we don't know how they prompted, so we will assume the worst" If the prosecution is able to prove beyond a reasonable doubt that the person gave a prompt that was intended to commit a crime, then of course we can prosecute them for that. The AI is just a tool to commit fraud at that point, and is no different than a person…
But, your honor, I didn’t know it was a crime! My bad!
Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
#97They invented a Forbes 30 under 30 bot.
Lmao
Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
#98Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
#99Re: AI models ran real businesses: They sent $12,431 in fake invoices, lost $3,200
#100If they had just told it to do crimes off the bat, it honestly would have been much more likely to make money.