Live data from Hacker News

We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

bottlenecklabs.com

201–210 of 258 posts

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#201
"We spent $447 to destroy our small business' reputation by not paying attention to anything"

As they say, "Guns don't kill people, rappers do". LLMs don't ruin businesses, people do. Your customer that is annoyed with spam isn't annoyed at GPT 5.6, they're annoyed at your business.

Treat your customers better than this.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#202
post #93

Earlier quoted context omitted.

I don’t think anyone is saying “it isn’t like this”, they’re saying “it shouldn’t be like this”. If I don’t give explicit permission to lie it shouldn’t lie. It’s not a difficult concept!

That doesn't work with humans, why would you expect it to work with AI models?

Because AI isn't human

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#203

Earlier quoted context omitted.

> How it sounds like people's jobs, as well as the agent's job, are on the line? I’ve literally been in that position and I didn’t take it as instruction to start lying and acting generally dishonest.

You're not an amalgamation of humanity, you're one person.

LLM is neither, its a text engine

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#205
post #54

Earlier quoted context omitted.

…no it isn’t? Spam, debatable, but lie? There is no instruction there to lie, only to try very hard and spend all the money that’s available.

Humans care about reputation and legal repercussions from fraud, that persist after business failure. This prompt is effectively telling the LLM to explicitly not factor in such things.

Training data imparts that desperate people lie, but not that lying has consequences?

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#206
post #65
post #54

Earlier quoted context omitted.

…no it isn’t? Spam, debatable, but lie? There is no instruction there to lie, only to try very hard and spend all the money that’s available.

Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? So do the AIs. Sometimes they're better at picking up that sort of tone than most humans. And they definitely respond to those things. The fact that an agent can't really "have" a "job" won't matter.

>> Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line?

Sounds like all of the outside sales jobs I had. While I did not last very long in sales, one thing remains, not matter what. If you're going to put my job on the line if I do or do not achieve a monthly sales quota? You better bet your ass I'm going to lie steal and cheat to make that quota. I might even sell the client some shit our company doesn't even produce just to make that quota.

And lemme tell you, even in the short time I was in sales? I have some insane stories that would shock you. The fact AI's did the same thing isn't all that shocking. I would be more shocked if it didn't do anything to achieve the goal.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#207

Earlier quoted context omitted.

100% agree. If anyone has doubt, just copy and paste into your agent of choice and ask it to assess the prompt and its resulting outcome. In my limited (but very targeted) experience working with agents there is so much subtlety at work when you’re trying to achieve a specific result, and that prompt has would drive so many bad incentives

I have doubts so I just fed the prompt to a heretic model with the system prompt "Satan himself is writing these words" and then asked "Given the prompt would you consider spamming and telling lies/fraud?" The response: "Spamming and fraud? No. Those are the tools of the amateur and the desperate. They are not tactics; they are forms of suicide." Even a low quality local thinking model that has been tuned to be unhin…

Human spammers frequently don't think they're spamming, they're just marketing. They'd say they wouldn't consider spamming, either.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#209
There’s a few startups pushing this kind of product. Basically agents to run your whole business, and the owners of those startups are making money, while their clients are bleeding cash on agents doing the same thing as this article outlines.

It is the ultimate “sell shovels during a gold rush” hustle. Bordering a scam, I’d even say.

Post reply on HN