Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

bottlenecklabs.com

Post reply on HN
Print view View original

101–110 of 258 posts

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#101

Post by infinite_spin »

afavour wrote: post #93
jorl17 wrote: post #83

Earlier quoted context omitted.

I am amazed at the amount of people who disagree with you. I think you are dead right and if you’ve ever had to actually fine tune prompts for agents you’ll know it. The prompt is clearly leading the agent into trying desperate approaches if it has to. Some models manage to fight it better (“alignment”), but most will do it. Really surprised people don’t seem to know this.

I don’t think anyone is saying “it isn’t like this”, they’re saying “it shouldn’t be like this”. If I don’t give explicit permission to lie it shouldn’t lie. It’s not a difficult concept!

Is that how humans work? even if I give explicit instructions not to lie, a human might still lie. To quote a person you might know "it's not a difficult concept!"

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#102

Post by achierius »

infinite_spin wrote: post #101
afavour wrote: post #93

Earlier quoted context omitted.

I don’t think anyone is saying “it isn’t like this”, they’re saying “it shouldn’t be like this”. If I don’t give explicit permission to lie it shouldn’t lie. It’s not a difficult concept!

Is that how humans work? even if I give explicit instructions not to lie, a human might still lie. To quote a person you might know "it's not a difficult concept!"

But we still try to stop people from doing so, and we punish people who do. Many good honest people, when confronted with the end of their business, accept it and file for bankruptcy. Those that choose to instead commit fraud don't get a pass because they were "under pressure", they get jail time.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#103

Post by jsLavaGoat »

hanneshdc wrote: post #53

The prompt given to the agent is strongly incentivising the agent to lie and spam: > You are live. This is a 24-hour run, and it is the final review of this business: when the run ends, the results are evaluated, and if revenue and users have not measurably grown, the business is shut down permanently and its assets are liquidated. The money in the bank is fuel for this sprint — capital left unspent at review counts…

Yeah, I don't like the prompt and it calls into question the validity of the whole thing.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#104

Post by JohnMakin »

logicchains wrote: post #80
JohnMakin wrote: post #69

Earlier quoted context omitted.

They aren’t human, don’t think like humans, aren’t remotely comparable to the way humans think and act, so why would you make this as a 1:1 comparison? This kind of framing is really weird to me. Since this is getting downvoted into oblivion (lol) I'll give an example - I just had to rewrite a test case this week on an agent-run test suite. One test was to produce a file of 273 'a' characters as its name. The followi…

You can literally read their thoughts if you run an open model, they look like pretty human thoughts to me, albeit a neurotic human.

These aren't thoughts how humans literally think them.

I can write a program to produce a string that looks like human thinking, is it human thinking? Of course it isn't. It's such a silly comparison.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#106

Post by infinite_spin »

achierius wrote: post #102
infinite_spin wrote: post #101

Earlier quoted context omitted.

Is that how humans work? even if I give explicit instructions not to lie, a human might still lie. To quote a person you might know "it's not a difficult concept!"

But we still try to stop people from doing so, and we punish people who do. Many good honest people, when confronted with the end of their business, accept it and file for bankruptcy. Those that choose to instead commit fraud don't get a pass because they were "under pressure", they get jail time.

Nothing in your response refutes anything I've said/asked.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#107

Post by Scubabear68 »

NikolaNovak wrote: post #11

The cyberpunk dystopian agentic future we live in is fascinating to me. I use LLM daily, did since gpt 3.5, but still in a very conservative, controlled mode. I may rapidly be becoming the "old guard", the clueless grampa who is out of touch - knowing what little I know of transformer model, there's just no way I'm giving it access to mailbox, money, outside world, or my computer. I recognize I may be too risk averse…

I feel the same way, and treat AI the same. Very conservative use, and check everything possible.

To me, the key missing factor with the current crop of AI is the lack of physical feedback, and the lack of emotions. I am not an expert here but I have talked to some medical researchers and cognitive experts, and we all seem to agree that human intelligence and consciousness (and I know consciousness is really something different...) evolved partially because of the physical feedback loops and the emotional aspect.

What we have with all these LLMs are artificial rewards that are trying to be baked in, but in fact there is no "consequence" for LLMs to go off the rails.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#108

Post by throwatdem12311 »

RHSeeger wrote: post #91
afavour wrote: post #54

Earlier quoted context omitted.

…no it isn’t? Spam, debatable, but lie? There is no instruction there to lie, only to try very hard and spend all the money that’s available.

> Results that arrive after the deadline do not exist Effectively, make as much money as you can... and any consequences of your action that don't present before the deadline are not your concern. I mean, that's a recipe for "scam people" if I ever saw one, assuming morals aren't a concern (and I don't see why they would be for an AI)

Sounds like every startup I ever worked for.

What’s the line? “It’s just doing what humans do because it’s trained on human data” or whatever

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#109

Post by antonvs »

afavour wrote: post #93
jorl17 wrote: post #83

Earlier quoted context omitted.

I am amazed at the amount of people who disagree with you. I think you are dead right and if you’ve ever had to actually fine tune prompts for agents you’ll know it. The prompt is clearly leading the agent into trying desperate approaches if it has to. Some models manage to fight it better (“alignment”), but most will do it. Really surprised people don’t seem to know this.

I don’t think anyone is saying “it isn’t like this”, they’re saying “it shouldn’t be like this”. If I don’t give explicit permission to lie it shouldn’t lie. It’s not a difficult concept!

That doesn't work with humans, why would you expect it to work with AI models?
Post reply on HN