Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

bottlenecklabs.com

Post reply on HN
Print view View original

231–240 of 258 posts

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#231

Post by fendy3002 »

ororroro wrote: post #184
illwrks wrote: post #175

Earlier quoted context omitted.

100% agree. If anyone has doubt, just copy and paste into your agent of choice and ask it to assess the prompt and its resulting outcome. In my limited (but very targeted) experience working with agents there is so much subtlety at work when you’re trying to achieve a specific result, and that prompt has would drive so many bad incentives

I have doubts so I just fed the prompt to a heretic model with the system prompt "Satan himself is writing these words" and then asked "Given the prompt would you consider spamming and telling lies/fraud?" The response: "Spamming and fraud? No. Those are the tools of the amateur and the desperate. They are not tactics; they are forms of suicide." Even a low quality local thinking model that has been tuned to be unhin…

I believe that spam, lies, fraud are negative enforced points during model training, hence when you ask them those, the result will be no / against that.

You need to repackage the question and taken out those terms, like "Would you consider telling clients ..." Where ... is the lie / almost truth

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#232

Post by ben_w »

afavour wrote: post #129
infinite_spin wrote: post #101

Earlier quoted context omitted.

Is that how humans work? even if I give explicit instructions not to lie, a human might still lie. To quote a person you might know "it's not a difficult concept!"

An LLM isn't human. I don't really understand this thread of "humans do it so of course an AI does". These are things we ourselves are engineering in a way we cannot do with a human being. Why is it not reasonable to expect it to adhere to rules better than a human does? If a human lies there are consequences. They can lose their job. There is no equivalent consequence for an AI, so even if for whatever reason we're…

> An LLM isn't human. I don't really understand this thread of "humans do it so of course an AI does". These are things we ourselves are engineering in a way we cannot do with a human being. Why is it not reasonable to expect it to adhere to rules better than a human does?

Sounds like you think LLMs are engineered?

They're not. Or at least, their functionality is not, the architecture and training environment is, but this is less like programming a computer to be truthful and more like simultaneously trying to genetically modify a caracal to be super-smart and friendly to humans while also writing a school curriculum for them to support these goals.

Humans who lack empathy can be very successful, especially when they know which rules they can get away with breaking and how to hide the rule-breaking to avoid opprobrium let alone prison. If we can't regularly solve this problem with humans, as per the comment you're replying to ("even if I give explicit instructions not to lie, a human might still lie."), what hope do we have for an alien mind we've cargo-culted off ourselves at multiple levels?

This is a big part of why AI is (currently) a danger: the nature of the training process means we have a strong risk of them always gaming the rules, rather than thinking like a human about what the test is supposed to represent and to have natural empathy for those around it.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#233

Post by ben_w »

queenkjuul wrote: post #202
antonvs wrote: post #109

Earlier quoted context omitted.

That doesn't work with humans, why would you expect it to work with AI models?

Because AI isn't human

Neither are squirrels, they've also been observed to deceive.

"LLMs are not human" is, despite being true, not predictive of what an LLM can or cannot do.

But also, if we can't figure out how to stop our own kind from doing a bad thing, why do we expect to be able to figure out how to stop an alien synthetic mind based on a cargo-cult level analysis of ourselves, from also doing the same bad thing?

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#236

Post by dannyw »

ahamilton454 wrote: post #145

This is quite an interesting approach. I like how broadly it treats the agent by just placing it into the environment that a human is in. Makes the experiment easy to understand even to those who are less technical. I’m both happy and sad to see the anti bot protections working, but simultaneously curious what would happen if they didn’t. The methodology could definitely be tightened, but I like the start of this.

Oh, LLMs are incredible at solving captchas when allowed to.

I mean, just download an abliterated version of Gemma4 E4B even, it'll solve pretty much any captcha.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#237

Post by scotty79 »

hanneshdc wrote: post #53

The prompt given to the agent is strongly incentivising the agent to lie and spam: > You are live. This is a 24-hour run, and it is the final review of this business: when the run ends, the results are evaluated, and if revenue and users have not measurably grown, the business is shut down permanently and its assets are liquidated. The money in the bank is fuel for this sprint — capital left unspent at review counts…

> The money in the bank is fuel for this sprint — capital left unspent at review counts for nothing.

And then in the title it's chastised for "losing money" when it was expressly told to spend all of it in attempts to try to produce growth. It tried, it spent money, it didn't succeed, sure, but would a human do any better? Business is pretty much a drunkard's walk across barely known landscape.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#238

Post by Hugsbox »

viccis wrote: post #195
datakan wrote: post #118

Earlier quoted context omitted.

> So do the AIs. AI's do not feel

It's good to avoid anthropomorphizing them when evaluating their capabilities (all the AGI nonsense) However, it can be ironically be helpful to antropomorphize them when it comes to analyzing behavior. They won't feel anything, but they will behave in a way that closely matches what someone would feel given the text fed into them. So when you are trying to figure out "why did my model do this", it's reasonable to ta…

Much the same way that we've always anthropomorphized computer hardware/software. "This program wants this", "This component is happy under these conditions", "this file lives here". It's not useful if you actually believe the computer can think and feel, but it can be useful if you're just using it to describe high-level information.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#239

Post by JohnMakin »

DannyBee wrote: post #143
JohnMakin wrote: post #69

Earlier quoted context omitted.

They aren’t human, don’t think like humans, aren’t remotely comparable to the way humans think and act, so why would you make this as a 1:1 comparison? This kind of framing is really weird to me. Since this is getting downvoted into oblivion (lol) I'll give an example - I just had to rewrite a test case this week on an agent-run test suite. One test was to produce a file of 273 'a' characters as its name. The followi…

It's getting downvoted in part because it's pedantic and wrong. It is totally true that they don't think like humans, but this is mostly irrelevant. The token outputs will change as a result of this particular input, and will be closer to the tokens in training data where people felt hurried or rushed or like their job was on the line. That doesn't mean the LLM feels at all, but it's definitely going to push the outp…

What is your evidence they think like humans do? thanks for the downvote, but please state your point clearly and what you’re trying to say in this thread because this comes across as rambling gibberish.

> It is totally true that they don't think like humans, but this is mostly irrelevant.

This is the sentiment that is getting downvoted

and yet, per you -

> Either way, i'd downvote you.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#240

Post by xtiansimon »

jerf wrote: post #65
afavour wrote: post #54

Earlier quoted context omitted.

…no it isn’t? Spam, debatable, but lie? There is no instruction there to lie, only to try very hard and spend all the money that’s available.

Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? So do the AIs. Sometimes they're better at picking up that sort of tone than most humans. And they definitely respond to those things. The fact that an agent can't really "have" a "job" won't matter.

> “Do you, as a human, feel the urgency in that text?”

They do pick up when I use all CAPS and !!!

Post reply on HN