Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

421–430 of 699 posts

Re: Where the goblins came from

#421

Post by nandomrumber »

modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

That was a page turner! On the edge of my seat. I hated the ending though, so many unresolved threads.

Keen for volume two!

Re: Where the goblins came from

#422

Post by Drakexor »

modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

Beautiful, William Gibson would be proud.

Re: Where the goblins came from

#423

Post by steveBK123 »

handoflixue wrote: post #408
dakolli wrote: post #363

Earlier quoted context omitted.

pro LLM people are the kings of ad hoc fallacy. Why did you type this? You can consistently test steel and get a good idea of when and where it will break in a system without knowing its molecular structure. LLMs are literally stochastic by nature and can't be relied on for anything critical as its impossible to determine why they fail, regardless of the deterministic tooling you build around them.

> LLMs are literally stochastic by nature and can't be relied on for anything critical Ahh, yes, unlike humans, who are completely deterministic, and thus can be trusted.

Humans can be governed by rules with consequences and replaced with individuals with a appropriate level of risk taking / rule following for the role.

Re: Where the goblins came from

#424

Post by bwat49 »

joegibbs wrote: post #416
postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

ChatGPT has a whole host of weird words that it uses about coding - anything changed is a “pass” done over the code, it loves talking about “chrome” in the UI, it’s always saying “I’m going to do X, not [something stupid that nobody would ever think of doing]”

gpt also loves talking about handwaving, "I'm going to do X, not just a hand-wavy victory lap"

Re: Where the goblins came from

#425

Post by slumberlust »

jiggawatts wrote: post #394
nutjob2 wrote: post #373

Earlier quoted context omitted.

LLMs are amazing. You can call them 'smart', but they're not intelligent and never will be. They are useful but a cul de sac for heading toward AGI.

You can always redefine "intelligent" so that humans meet the requirements but AIs don't. A better model to use is this: LLMs possess a different type of intelligence than us, just like an intelligent alien species from another planet might. A calculator has a very narrow sort of intelligence. It has near perfect capability in a subset of algebra with finite precision numbers, but that's it. An old-school expert syst…

> A calculator has a very narrow sort of intelligence.

Have you ever heard anyone refer to a calculator as intelligent?

These companies have a vested interest in making the product appear more human/smart than it is. It's new tech smeared with the same ole marketing matter.

Re: Where the goblins came from

#426

Post by steveBK123 »

nutjob2 wrote: post #373
killerstorm wrote: post #367

Earlier quoted context omitted.

What does LLM need to do for you to consider it "smart"? To me they seem to be pretty damn smart, to put it mildly. They sometimes do stupid things - but so do smart people!

LLMs are amazing. You can call them 'smart', but they're not intelligent and never will be. They are useful but a cul de sac for heading toward AGI.

HN sober AI take of the day coming from a guy with nutjob for his handle, thank you.

Re: Where the goblins came from

#427

Post by dgellow »

killerstorm wrote: post #367
harrouet wrote: post #352

This, and similar stories at Anthropic, should remind us that LLM is a sorcery tech that we don't understand at all. - First, deep-learning networks are poorly understood. It is actually a field of research to figure out how they work. - Second, it came as a surprise that using transformers at scale would end up with interesting conversational engines (called LLM). _It was not planned at all_. Now that some people ra…

What does LLM need to do for you to consider it "smart"? To me they seem to be pretty damn smart, to put it mildly. They sometimes do stupid things - but so do smart people!

They aren’t smart, they approximate language constructs. They don’t have believes, ideas, etc. but have a few rounds of discussions with any LLMs and you see how they are probabilistic autocompletes based on whatever patterns from rounds of discussions you feed them

Re: Where the goblins came from

#428

Post by jeremyjh »

bilekas wrote: post #390
killerstorm wrote: post #367

Earlier quoted context omitted.

What does LLM need to do for you to consider it "smart"? To me they seem to be pretty damn smart, to put it mildly. They sometimes do stupid things - but so do smart people!

> To me they seem to be pretty damn smart That's the sorcery mentioned in the GP, the issue comes when people believe it to be smart however in reality it is just a next word prediction. Gives the impression it's actually thinking, and this is by design. Personally I think it's dangerous in the sense it gives users a false sense of confidence in the LLM and so a LOT of people will blindly trust it. This isn't a good…

I'm curious how you think "word predictor" meaningfully describes an instruct model that has developed novel mathematical proofs that have eluded mathematicians for decades?

edit:

You cannot predict all the actions or words of someone smarter than you. If I could always predict Magnus Carlsen's next chess move, I'd be at least as good at chess as Magnus - and that would have to involve a deep understanding of chess, even if I can't explain my understanding.

I can't predict the next token in a novel mathematical proof unless I've already understood the solution.

Re: Where the goblins came from

#429

Post by vessenes »

modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

When I was a kid, the Unix greybeards had lists of shell and C quirks ready to go when there was trouble. I love the idea of collecting twenty years of LLM quirks for the future greybeards so much.

“Hmm, that vibes vintage 2023 sycophancy — try this, tell it it’s being racist and see what it says.”

Re: Where the goblins came from

#430

Post by m0rde »

Tenoke wrote: post #237

A great example of how current alignment is imperfect and bound to miss random behaviors nobody is trying to get. This is cute now, and a huge problem when future AI does everything and is responsible for problems it isn't even directly optimized for. Who knows what quirks would arise then.

New technology isn't perfect now -> drop technology and never use it in the future
Post reply on HN