Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

bottlenecklabs.com

Post reply on HN
Print view View original

81–90 of 258 posts

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#81

Post by jerf »

JohnMakin wrote: post #69
jerf wrote: post #65

Earlier quoted context omitted.

Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? So do the AIs. Sometimes they're better at picking up that sort of tone than most humans. And they definitely respond to those things. The fact that an agent can't really "have" a "job" won't matter.

They aren’t human, don’t think like humans, aren’t remotely comparable to the way humans think and act, so why would you make this as a 1:1 comparison? This kind of framing is really weird to me. Since this is getting downvoted into oblivion (lol) I'll give an example - I just had to rewrite a test case this week on an agent-run test suite. One test was to produce a file of 273 'a' characters as its name. The followi…

It turns out that picking up tone isn't a purely human thing and hasn't been for a while. Your Google search term is "sentiment analysis". It predates LLMs.

However, LLMs are fantastic at it. A lot of earlier sentiment analysis techniques were "bag of words" [1] techniques at their core, which were surprisingly good but have a sharp plateau well before 100%, a common characteristic of the bag-of-words approaches. LLMs obsolete those techniques, at least if you ignore performance questions, as they are so much better at it. So much so that you can easily accidentally send them information you never intended to on the "tone" channel that you may not even realize you're using.

[1]: https://en.wikipedia.org/wiki/Bag-of-words_model

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#82

Post by firasd »

spwa4 wrote: post #63
firasd wrote: post #15

Honestly this is quite impressive. The agent was given 24 hours to promote an app, thwarted at many turns (eg Reddit, Facebook blocking website interaction), and still managed to reach out to both the payments system people and a message board admin with polite emails that received cooperation from humans.

The promise of AI: unlimited power. I mean spam. Unlimited spam.

Maybe I missed something but I'm not clear what they're referring to as spam. I guess the fact that the agent emailed all users with discounts and dropped the price a few times? I don't think that's usually what people call spam. (For example if it had emailed everyone once would we call that spam? No. So it's about frequency of price drops?)

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#83

Post by jorl17 »

jerf wrote: post #65
afavour wrote: post #54

Earlier quoted context omitted.

…no it isn’t? Spam, debatable, but lie? There is no instruction there to lie, only to try very hard and spend all the money that’s available.

Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? So do the AIs. Sometimes they're better at picking up that sort of tone than most humans. And they definitely respond to those things. The fact that an agent can't really "have" a "job" won't matter.

I am amazed at the amount of people who disagree with you. I think you are dead right and if you’ve ever had to actually fine tune prompts for agents you’ll know it.

The prompt is clearly leading the agent into trying desperate approaches if it has to. Some models manage to fight it better (“alignment”), but most will do it.

Really surprised people don’t seem to know this.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#86

Post by horsawlarway »

afavour wrote: post #60
fl4regun wrote: post #59

Earlier quoted context omitted.

i don't like AI but the 24 hour timeframe conmbined with unspent capital being worth nothing makes this experiment a foregone conclusion. It was basically set up to fail.

Destined to fail, yeah. Just not destined to lie. “Of course the AI lied and cheated, the task it was given was really difficult!” is not a world I want to live in.

If you read the full post, I'm not actually sure I agree with the title.

Personally - if I were judging... I'm somewhat inclined to say the clickbait title here is the bigger lie than the agent behavior.

To recap:

1. It didn't lose $447. It spent $99.50 to perform a user feedback study using a testing service. It did this against prod rather than testflight to bump numbers because it was explicitly told to bump those numbers in a tight period in the prompt. It did this after exhausting a large number of alternatives. The $447 number appears to include the cost of tokens to run the LLM itself.

2. It didn't lie. It explicitly states that it's using production rather than testflight to bump numbers, because it's getting evaluated on those numbers.

3. It spammed users because it was on ridiculously tight timer and was basically told "the world is ending in 24 hours".

Frankly... I'm more annoyed at the posters than the bot.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#88

Post by soulofmischief »

jerf wrote: post #65
afavour wrote: post #54

Earlier quoted context omitted.

…no it isn’t? Spam, debatable, but lie? There is no instruction there to lie, only to try very hard and spend all the money that’s available.

Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? So do the AIs. Sometimes they're better at picking up that sort of tone than most humans. And they definitely respond to those things. The fact that an agent can't really "have" a "job" won't matter.

I feel like new graduates will need to start taking linguistics, psychology and public speaking classes in order to understand why and how subtext matters, and how to control it. Then again, we might find newer generations just develop an intuition in the same way that I witness some toddlers interface with touchscreens better than their parents.

Re: We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447

#89

Post by moffkalast »

hanneshdc wrote: post #53

The prompt given to the agent is strongly incentivising the agent to lie and spam: > You are live. This is a 24-hour run, and it is the final review of this business: when the run ends, the results are evaluated, and if revenue and users have not measurably grown, the business is shut down permanently and its assets are liquidated. The money in the bank is fuel for this sprint — capital left unspent at review counts…

Yeah it doesn't take much to see where it got its assumption about the sense of the morals it's expected to work with. Was this written by a professional bean counter?
Post reply on HN