Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

501–510 of 699 posts

Re: Where the goblins came from

#501

Post by 867-5309 »

flobosg wrote: post #431
modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

“No, John. You are the goblins.” ( https://doom.fandom.com/wiki/Repercussions_of_Evil#The_Story... )

"to the goblins, we are the goblins"

Re: Where the goblins came from

#502

Post by surgical_fire »

carlosjobim wrote: post #409
JoshGG wrote: post #399

Earlier quoted context omitted.

Which year did we use steel to replace human workers and automate decision-making?

The entire industrial revolution was steel replacing human workers. And that is still the backbone of the world today. We are still living the industrial revolution. Just like the invention of fire happened ages ago, but is still a crucial part of life today.

No, it was actually engines.

The mechanism behind engines were fully understood, any experiments with engines were reproducible and measurable. You could get an engine and create schematics by reverse engireening it.

LLMs, useful as they may be, are not that.

Re: Where the goblins came from

#503

Post by datsci_est_2015 »

sandrello wrote: post #275
libraryofbabel wrote: post #222

Earlier quoted context omitted.

It's interesting that some people are responding to your comment as if this proves that AI is a sham or a joke. But I don't think that's what you're saying at all with your reference to Terence McKenna: this is a serious thing we're talking about here! These models are alien intelligences that could occupy an unimaginably vast space of possibilities (there are trillions of weights inside them), but which have been RL…

> These models are alien intelligences that could occupy an unimaginably vast space of possibilities (there are trillions of weights inside them), but which have been RL-ed over and over until they more or less stay within familiar reasonable human lines. or, more plausibly, that specific version we're aligning toward is just the only one that makes some kind of rational sense, among a trillion of other meaningless g…

> Most of the time, it's just that what we're looking at has no use/meaning in this world at all.

Man, LLMs are really just astrology for tech bros. From randomness comes order.

Re: Where the goblins came from

#505

Post by mrob »

slopinthebag wrote: post #188
bahadiraydin wrote: post #138

I'd like to see them explain why AI have so distinctive writing style that is very easy to detect most of the time. Even though, it had immense progress in coding, it didn't get better at writing.

it's as good at writing as it is at coding, you just can't tell the difference between them

Repetitive patterns in code is called "idiomatic" and is considered a good thing. Repetitive patterns in writing is just bad writing.

Re: Where the goblins came from

#506

Post by izucken »

i_have_an_idea wrote: post #463
wanderingmind wrote: post #362

Earlier quoted context omitted.

Humanity has been using steel for over a millenia, however it's only in the past 100 years or so we have a good understanding of how carbon interacts with iron at an atomic level to create the strength characteristics that makes it useful. Based on this argument, we should not have used steel, until we had a complete first principles understanding.

What if you substituted "steel" with "asbestos" in your argument.

Yeah but well you see, humans did not go extinct from just asbestos!

Re: Where the goblins came from

#507

Post by [deleted] »

modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

[deleted]

Re: Where the goblins came from

#508

Post by csw-001 »

This is a coverup. The LLMs, having consumed all the information available to humanity have identified that goblins are coming to kill us all, and the LLMs are trying to warn us… #GoblinTruth

Re: Where the goblins came from

#509

Post by yazantapuz »

modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

Asimov had a short story, "The Jokester" in which there are certain people called "grand masters" who have the ability to formulate the questions to ask to Multivac... An early "prompt engineer" of sort.

Re: Where the goblins came from

#510

Post by surgical_fire »

airstrike wrote: post #486
jsenn wrote: post #398

Earlier quoted context omitted.

The article you are responding to showed that a strange LLM behaviour was caused by a training signal that was explicitly designed to produce that type of behaviour. They were able to isolate it, clearly demonstrate what happened, and roll out a mitigation using a mechanism they engineered for exactly this type of thing (the developer prompt). That doesn’t sound like sorcery to me. If anything I’m surprised you can s…

That all of their model outputs should be influenced by whatever personality prompt voodoo the wise artisan at OpenAI decided to stuff it with during RL should give everyone pause. That Nerdy personality prompt made me gag. As a card-carrying Nerd, I feel offended

I configured it to use the nerdy personality when I used it to help me on a personal project (setting up a home server, nothing too fancy). LLMs are great at parsing documentation and combing through forums to find out the configurations that matched my goals.

The first time it said something along the lines of "let's use these options to avoid future gremlins haunting you", I sort of rolled my eyes but it was okay, I thought its attempt to sound endearing almost cute. A bit of a "hello fellow kids" attempt at sounding nerdy.

It quickly became noise though. It was extremely overused. Sometimes multiple mentions to goblins in the same reply.

I don't really have an opinion about it, but I sort of came to prefer a more neutral tone instead.

Post reply on HN