Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

461–470 of 699 posts

Re: Where the goblins came from

#461

Post by lxgr »

dgellow wrote: post #427
killerstorm wrote: post #367

Earlier quoted context omitted.

What does LLM need to do for you to consider it "smart"? To me they seem to be pretty damn smart, to put it mildly. They sometimes do stupid things - but so do smart people!

They aren’t smart, they approximate language constructs. They don’t have believes, ideas, etc. but have a few rounds of discussions with any LLMs and you see how they are probabilistic autocompletes based on whatever patterns from rounds of discussions you feed them

At what point does autocomplete stop being "just autocomplete"?

Clearly there's a limit. For example, if an alien autocomplete implementation were to fall out of a wormhole that somehow manages to, say, accurately complete sentences like "S&P 500, :" with tomorrow's actual closing value today, I'd call that something else.

Re: Where the goblins came from

#462

Post by qwery »

wanderingmind wrote: post #362
harrouet wrote: post #352

This, and similar stories at Anthropic, should remind us that LLM is a sorcery tech that we don't understand at all. - First, deep-learning networks are poorly understood. It is actually a field of research to figure out how they work. - Second, it came as a surprise that using transformers at scale would end up with interesting conversational engines (called LLM). _It was not planned at all_. Now that some people ra…

Humanity has been using steel for over a millenia, however it's only in the past 100 years or so we have a good understanding of how carbon interacts with iron at an atomic level to create the strength characteristics that makes it useful. Based on this argument, we should not have used steel, until we had a complete first principles understanding.

Assuming your timeline and metallurgical claims to be true, you're conflating engineering and (materials) science.

Humans have been using steel for however long, when and where it was understood to be an appropriate solution to a problem. In some sense, engineering is the development and application of that understanding. You do not need to have a molecular explanation of the interaction between carbon and iron to do effective engineering[-1] with steel.[0] Science seeks to explain how and why things are the way they are, and this can inform engineering, but it is not prerequisite.

I think that machine learning as a field has more of an understanding of how LLMs work than your parent post makes out. But I agree with the thrust of that comment because it's obvious that the reckless startups that are pushing LLMs as a solution to everything are not doing effective engineering.

[-1] "effective engineering" -- that's getting results, yes, but only with reasonable efficiency and always with safety being a fundamental consideration throughout

[0] No, I'm not saying that every instance of the use of steel has been effective/efficient/safe.

Re: Where the goblins came from

#463

Post by i_have_an_idea »

wanderingmind wrote: post #362
harrouet wrote: post #352

This, and similar stories at Anthropic, should remind us that LLM is a sorcery tech that we don't understand at all. - First, deep-learning networks are poorly understood. It is actually a field of research to figure out how they work. - Second, it came as a surprise that using transformers at scale would end up with interesting conversational engines (called LLM). _It was not planned at all_. Now that some people ra…

Humanity has been using steel for over a millenia, however it's only in the past 100 years or so we have a good understanding of how carbon interacts with iron at an atomic level to create the strength characteristics that makes it useful. Based on this argument, we should not have used steel, until we had a complete first principles understanding.

What if you substituted "steel" with "asbestos" in your argument.

Re: Where the goblins came from

#464

Post by hansmayer »

wanderingmind wrote: post #362
harrouet wrote: post #352

This, and similar stories at Anthropic, should remind us that LLM is a sorcery tech that we don't understand at all. - First, deep-learning networks are poorly understood. It is actually a field of research to figure out how they work. - Second, it came as a surprise that using transformers at scale would end up with interesting conversational engines (called LLM). _It was not planned at all_. Now that some people ra…

Humanity has been using steel for over a millenia, however it's only in the past 100 years or so we have a good understanding of how carbon interacts with iron at an atomic level to create the strength characteristics that makes it useful. Based on this argument, we should not have used steel, until we had a complete first principles understanding.

Oh for crying out loud! Let's stop inventing fake analogies to justify the inherent LLM shortcomings! Those of us who are critical - are only using the standards that the LLM companies set themselves ("superintelligence", "pocket phds" bla blabla), to hold them accountable. When does the grift stop?

Re: Where the goblins came from

#465

Post by cryptopian »

thatguymike wrote: post #114
nomilk wrote: post #4

> We unknowingly gave particularly high rewards for metaphors with creatures. I recall a math instructor who would occasionally refer to variables (usually represented by intimidating greek letters) as "this guy". Weirdly, the casual anthropomorphism made the math seem more approachable. Perhaps 'metaphors with creatures' has a similar effect i.e. makes a problem seem more cute/approachable. On another note, buzzword…

A decade ago I gave a presentation on automata theory. I demonstrated writing arbitrary symbols to tape with greek letters, just like I’d learned at university. The audience was pretty confused and didn’t really grok the presentation. A genius communicator in the audience advised me to replace the greek letters with emoji… I gave the same presentation to the same demographic audience a week later and it was a smash h…

Most human brains just aren't very good at coping with abstract concepts. It reminds me of the Wason selection task[1]. You give participants a formal logic problem to solve, "how many cards do you have to turn over to show that the rules are being followed". If the rule is "a card with a vowel on one side _must_ have an even number on the other", people do very badly making illogical assumptions. If the rule is "one side has a bar order, and the other side has the age of the person making the order. The person must be above the legal age", it makes sense and people do well, because we understand bars, drinks and the laws thereof.

[1] https://en.wikipedia.org/wiki/Wason_selection_task

Re: Where the goblins came from

#466

Post by creamyhorror »

afro88 wrote: post #410
postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

> The obsession with the word "seam" as it pertains to coding I quite liked this term when it started using it. And I appreciate the consistent way it talks about coding work even when working on radically different stacks and codebases

"Seam" has been stretched by AI from its original legacy-code context to any point in code where something can be plugged in. I actually asked an AI about this a few weeks ago because I was surprised by the consistent, frequent use of "seam".

Frequent words I see from GPT: "shape", "seam", "lane", "gate" (especially as verb), "clean", "honest", "land", "wire", "handoff", "surface" (noun), "(un)bounded", "semantics" (but this one is fair enough), and sometimes "unlock"

It feels like AI really likes to pick the shortest ways to express ideas even if they aren't the most common, which I suppose would make sense if that's actually what's happening.

Re: Where the goblins came from

#467

Post by hansmayer »

handoflixue wrote: post #408
dakolli wrote: post #363

Earlier quoted context omitted.

pro LLM people are the kings of ad hoc fallacy. Why did you type this? You can consistently test steel and get a good idea of when and where it will break in a system without knowing its molecular structure. LLMs are literally stochastic by nature and can't be relied on for anything critical as its impossible to determine why they fail, regardless of the deterministic tooling you build around them.

> LLMs are literally stochastic by nature and can't be relied on for anything critical Ahh, yes, unlike humans, who are completely deterministic, and thus can be trusted.

Wow, such a nasty view to hold. What's next, the Altman's bullshit argument about "all the food" that the humans need to grow up and develop brain ? Humans are intelligent. Humans can generalise and invent new concepts, ideas and art. LLMs are none of that.

Re: Where the goblins came from

#468

Post by steve1977 »

killerstorm wrote: post #367
harrouet wrote: post #352

This, and similar stories at Anthropic, should remind us that LLM is a sorcery tech that we don't understand at all. - First, deep-learning networks are poorly understood. It is actually a field of research to figure out how they work. - Second, it came as a surprise that using transformers at scale would end up with interesting conversational engines (called LLM). _It was not planned at all_. Now that some people ra…

What does LLM need to do for you to consider it "smart"? To me they seem to be pretty damn smart, to put it mildly. They sometimes do stupid things - but so do smart people!

Are they smart or are they imitating things smart people did? (and if so, is there a difference?)

Re: Where the goblins came from

#469

Post by rjmill »

dummydummy1234 wrote: post #434
modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

So, I always thought that Warhammer 40k techpriests were absurd. Strange obscure religious rituals to appease the machine spirit. But at this point I can actually see something like that. What is prompt engineering but a strange pseudo ritual. So praise the Omnissiah, I guess...

They've always resonated with me, maybe because I often work on legacy code. All this ancient technology that no one understands. Crazy rituals/incantations to get things done. People being afraid to skip steps, even if it probably isn't needed. The aversion to unconsecrated (non IT-supported) technology.

The machine spirits were the only part that felt "too magical" to me, but now we're well on our way. The Omnissiah's blessings be upon us.

(Let's just skip servitors. Those give me the heebie-jeebies.)

Re: Where the goblins came from

#470

Post by ashtonshears »

modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

LOL
Post reply on HN