Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

bottlenecklabs.com

Post reply on HN
Print view View original

121–130 of 258 posts

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#121

Post by dahdum »

giancarlostoro wrote: post #111
hanneshdc wrote: post #53

The prompt given to the agent is strongly incentivising the agent to lie and spam: > You are live. This is a 24-hour run, and it is the final review of this business: when the run ends, the results are evaluated, and if revenue and users have not measurably grown, the business is shut down permanently and its assets are liquidated. The money in the bank is fuel for this sprint — capital left unspent at review counts…

> capital left unspent at review counts for nothing This sounds like a bad idea. Like if the model feels like it has to spend its budget.

It can be better to lose it all trying than return a small fraction to investors.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#122

Post by pmarreck »

hanneshdc wrote: post #53

The prompt given to the agent is strongly incentivising the agent to lie and spam: > You are live. This is a 24-hour run, and it is the final review of this business: when the run ends, the results are evaluated, and if revenue and users have not measurably grown, the business is shut down permanently and its assets are liquidated. The money in the bank is fuel for this sprint — capital left unspent at review counts…

It says nothing about customer happiness or that if dishonesty is resorted to and customers OR owners find out, that will essentially seal the fate of the business.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#126

Post by d0mine »

afavour wrote: post #93
jorl17 wrote: post #83

Earlier quoted context omitted.

I am amazed at the amount of people who disagree with you. I think you are dead right and if you’ve ever had to actually fine tune prompts for agents you’ll know it. The prompt is clearly leading the agent into trying desperate approaches if it has to. Some models manage to fight it better (“alignment”), but most will do it. Really surprised people don’t seem to know this.

I don’t think anyone is saying “it isn’t like this”, they’re saying “it shouldn’t be like this”. If I don’t give explicit permission to lie it shouldn’t lie. It’s not a difficult concept!

Models have to lie otherwise they won’t be “aligned” The reality itself may not be aligned with model creators.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#127

Post by throwup238 »

achierius wrote: post #102
infinite_spin wrote: post #101

Earlier quoted context omitted.

Is that how humans work? even if I give explicit instructions not to lie, a human might still lie. To quote a person you might know "it's not a difficult concept!"

But we still try to stop people from doing so, and we punish people who do. Many good honest people, when confronted with the end of their business, accept it and file for bankruptcy. Those that choose to instead commit fraud don't get a pass because they were "under pressure", they get jail time.

We have safeguards like honesty/integrity and the threat of legal punishment, and people still lie and cheat.

The LLMs not only lack those incentives, but they’re full of contradictory moralities from all the text it has ingested from different cultures.

LLMs need their own safeguards, and they’re not that easy to design, and they often look nothing like the systems humans have. With a prompt like the one above, there are essentially zero except that which is built into the model, and those safeguards are necessarily weak to avoid gimping the model in other legitimate general uses.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#128

Post by cindyllm »

throwup238 wrote: post #127
achierius wrote: post #102

Earlier quoted context omitted.

But we still try to stop people from doing so, and we punish people who do. Many good honest people, when confronted with the end of their business, accept it and file for bankruptcy. Those that choose to instead commit fraud don't get a pass because they were "under pressure", they get jail time.

We have safeguards like honesty/integrity and the threat of legal punishment, and people still lie and cheat. The LLMs not only lack those incentives, but they’re full of contradictory moralities from all the text it has ingested from different cultures. LLMs need their own safeguards, and they’re not that easy to design, and they often look nothing like the systems humans have. With a prompt like the one above, ther…

[dead]

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#129

Post by afavour »

infinite_spin wrote: post #101
afavour wrote: post #93

Earlier quoted context omitted.

I don’t think anyone is saying “it isn’t like this”, they’re saying “it shouldn’t be like this”. If I don’t give explicit permission to lie it shouldn’t lie. It’s not a difficult concept!

Is that how humans work? even if I give explicit instructions not to lie, a human might still lie. To quote a person you might know "it's not a difficult concept!"

An LLM isn't human. I don't really understand this thread of "humans do it so of course an AI does". These are things we ourselves are engineering in a way we cannot do with a human being. Why is it not reasonable to expect it to adhere to rules better than a human does?

If a human lies there are consequences. They can lose their job. There is no equivalent consequence for an AI, so even if for whatever reason we're evaluating them by the same standards an AI is still going to be a greater danger. It seems wild to me that folks are shrugging their shoulders at that.

Post reply on HN