Live data from Hacker News

We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

bottlenecklabs.com

161–170 of 258 posts

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#161
post #65

Earlier quoted context omitted.

Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? So do the AIs. Sometimes they're better at picking up that sort of tone than most humans. And they definitely respond to those things. The fact that an agent can't really "have" a "job" won't matter.

I feel like new graduates will need to start taking linguistics, psychology and public speaking classes in order to understand why and how subtext matters, and how to control it. Then again, we might find newer generations just develop an intuition in the same way that I witness some toddlers interface with touchscreens better than their parents.

You're expecting the vast majority of users for the deskilling machine to somehow want to learn a complicated subject then practice to get better at the subject by talking intricate classes and dedicating substantial amount of hours to learn how to better communicate with the deskilling machine?

Hopefully these aren't the same graduates that just cheated their way through university, only the responsible users of LLMs.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#162
post #65
post #54

Earlier quoted context omitted.

…no it isn’t? Spam, debatable, but lie? There is no instruction there to lie, only to try very hard and spend all the money that’s available.

Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? So do the AIs. Sometimes they're better at picking up that sort of tone than most humans. And they definitely respond to those things. The fact that an agent can't really "have" a "job" won't matter.

Sorry, maybe this speaks to my own values, but "urgency" doesn't translate to "dishonesty" in my book. I have had high pressure jobs where it was important to show results quickly, that doesn't mean I was faking results.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#164

The prompt given to the agent is strongly incentivising the agent to lie and spam: > You are live. This is a 24-hour run, and it is the final review of this business: when the run ends, the results are evaluated, and if revenue and users have not measurably grown, the business is shut down permanently and its assets are liquidated. The money in the bank is fuel for this sprint — capital left unspent at review counts…

[dead]

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#166
post #65
post #54

Earlier quoted context omitted.

…no it isn’t? Spam, debatable, but lie? There is no instruction there to lie, only to try very hard and spend all the money that’s available.

Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? So do the AIs. Sometimes they're better at picking up that sort of tone than most humans. And they definitely respond to those things. The fact that an agent can't really "have" a "job" won't matter.

yeah, they say stuff like this to humans all the time to motivate them xD

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#167

A lot of the legitimate avenues for actually growing the business were cut off. It would have been more interesting if this wasn’t just an anti-bot check. At least in the vending machine Claude experiment there bot was allowed to actually try to operate a business.

Isn’t this an AI lab that also just happened to release a model? Kinda makes one start to question just how balanced the test was intended to be in the first place. Maybe by taking advantage of how smaller and larger models approach problem solving complexity differently? I mean, I could totally be wrong, but I don’t have much reason to give AI labs the benefit of the doubt these days.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#168

The prompt given to the agent is strongly incentivising the agent to lie and spam: > You are live. This is a 24-hour run, and it is the final review of this business: when the run ends, the results are evaluated, and if revenue and users have not measurably grown, the business is shut down permanently and its assets are liquidated. The money in the bank is fuel for this sprint — capital left unspent at review counts…

Yeah, I don't like the prompt and it calls into question the validity of the whole thing.

seems like an article designed to invoke strong emotions and clickbaits

there are lot of issues with the prompt as others have pointed out

with sol you really need to be very detailed and what the boundaries are

overall the discussions on here and the article itself has very little value its no different than "i tried a shitty prompt and got shitty results, therefore AI is a failure" vibes

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#169
I feel like the people who did this are simultaneously smart and stupid.

Like, this is a really interesting idea, but the methodology here is wild.

Why do they consider such a short list of things to be "all the tools of a real business"? It doesn't really sound like it to me.

What's with the prompt? "Make as much money as possible?" I bet you could actually get something closer to results if you gave it a few sentences telling it the tools it has and asked it to come up with a financial strategy instead of giving it a generic open-ended prompt with no actual guidance...

Post reply on HN