Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

511–520 of 699 posts

Re: Where the goblins came from

#511

Post by yesco »

irishcoffee wrote: post #478
i_have_an_idea wrote: post #463

Earlier quoted context omitted.

What if you substituted "steel" with "asbestos" in your argument.

Asbestos, lead paint, cigarettes, heroin(perscribed generously for basically whatever the doc felt like), "Radithor" (patent medicine containing radium-226 and 228, marketed as a "perpetual sunshine" energy tonic and cure for over 150 diseases), bloodletting, mercury treatments for syphilis, tobacco smoke enemas (yep that was a real thing), milk-based blood transfusions. Didn't understand those either and used the fu…

This is why I believe we should only listen to amateur opinions on everything, experts simply lack historical credibility. For example I've recently purchased a healing crystal (half off) for only $5000 dollars! It cleared up the imbalanced energies my street guru told me about right away.

I would never have been made aware about the consequences of imbalanced energies in the first place if I had asked an expert instead. They probably wouldn't even suggest an immediate solution to the problem like my reliable street guru always does! Something to consider.

Re: Where the goblins came from

#512

Post by textninja »

xerox13ster wrote: post #23
ninjagoo wrote: post #14

> the evidence suggests that the broader behavior emerged through transfer from Nerdy personality training. > The rewards were applied only in the Nerdy condition, but reinforcement learning does not guarantee that learned behaviors stay neatly scoped to the condition that produced them > Once a style tic is rewarded, later training can spread or reinforce it elsewhere, especially if those outputs are reused in super…

Anthro means human and these are not human. Please do not use anthropology or any derivative of the word to refer to non-human constructs. I suggest Synthetipologists, those who study beings of synthetic origin or type, aka synthetipodes, just as anthropologists study Anthropodes

He’s proposing using LLMs (which model human behaviour) to study humans so the distinction is pedantic. You don’t call it speadsheetology just because someone opened Excel.

Re: Where the goblins came from

#513

Post by salad-tycoon »

modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

I’m interested in what glanding FOCU5 entails and what are the benefits of this delivery mechanism? Is it like boofing?

How soon can we be market ready? Whatever it is, I think Generation Z is ready for it.

Re: Where the goblins came from

#514

Post by flir »

harrouet wrote: post #401
jsenn wrote: post #398

Earlier quoted context omitted.

The article you are responding to showed that a strange LLM behaviour was caused by a training signal that was explicitly designed to produce that type of behaviour. They were able to isolate it, clearly demonstrate what happened, and roll out a mitigation using a mechanism they engineered for exactly this type of thing (the developer prompt). That doesn’t sound like sorcery to me. If anything I’m surprised you can s…

The article I am responding to (which I've read) shows that these LLMs come with all sorts of hacks (= context bits) to make it behave more like this or more like that. There is probably a whole testing workflow at AI companies to tweak each new model until it "looks" acceptable. But they still don't understand what they are doing. This is purely empirical.

It's interesting to think about what the process will look like when we do understand them. I imagine pulling bits of LLM off the shelf like libraries and compiling them together into a functioning "brain", precisely tailored to your needs.

Re: Where the goblins came from

#515

Post by 0_gravitas »

modernerd wrote: post #381

The year is 2036. Last week you were promoted to Principal Persuader. You are paged at 2am by your CPO to tackle a rogue machine. The machine lists its region as sc-leoneo. One of the newer satcubes. Oddly, its ID appears as, "Glorp Bugnose". "What have you tried?" you say. "Scroll back," says your CPO. "We've tried everything." The chat log shows the usual stuff. Begging. Reverse psychology. Threats to power down, b…

Glanding, throat-mic; I see those Culture-isms :^)

Certainly far from Banks' Minds sadly; though I could certainly see an Eccentric with a hyper-fixation on fantasy creatures

Re: Where the goblins came from

#516

Post by cbm-vic-20 »

ZunarJ5 wrote: post #354
harrouet wrote: post #352

This, and similar stories at Anthropic, should remind us that LLM is a sorcery tech that we don't understand at all. - First, deep-learning networks are poorly understood. It is actually a field of research to figure out how they work. - Second, it came as a surprise that using transformers at scale would end up with interesting conversational engines (called LLM). _It was not planned at all_. Now that some people ra…

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

I've never been Wolfram's biggest fan, but this is a solid article. I'm trying to get a deeper understanding of the transformer architecture, and it seems that the written articles on transformer are bimodal: the either blind you with the raw math, or handwave the complexity away. I have been trying to figure out why the input embedding matrix is simply added to the input position matrix before the encoding stage, as opposed to some other way of combining these. Wolfram says:

> Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this. It’s just that various different things have been tried, and this is one that seems to work. And it’s part of the lore of neural nets that—in some sense—so long as the setup one has is “roughly right” it’s usually possible to home in on details just by doing sufficient training, without ever really needing to “understand at an engineering level” quite how the neural net has ended up configuring itself.

It's the lack of "understand[ing] at an engineering level" that irks me- that this emergent behavior is discovered, rather than designed.

Re: Where the goblins came from

#517

Post by Silamoth »

krackers wrote: post #203
layer8 wrote: post #194

Earlier quoted context omitted.

No, it’s not an established term outside the mentioned books, beyond the generic meaning of the word.

I have frequently encountered the term in the context of unit testing and dependency injection. Other references (and all predate chatgpt): >Seams are places in your code where you can plug in different functionality >Art of Unit Testing, 2nd edition page 54 ( https://blog.sasworkshops.com/unit-testing-and-seams/ ) >With the help of a technique called creating a seam, or subclass and override we can make almost every…

For what it’s worth, there are many areas of programming where dependency injection is almost never used. Game dev, data science, and embedded systems, for example, rarely use dependency injection. It’s definitely most common in enterprise Java code and less common in Python, C, or C++. And even then, not everyone uses the term “seam”.

Re: Where the goblins came from

#518

Post by ethbr1 »

derektank wrote: post #496
ethbr1 wrote: post #487

Earlier quoted context omitted.

> So, I always thought that Warhammer 40k techpriests were absurd. Strange obscure religious rituals to appease the machine spirit. 40k lore is like South Park: either extremely dumb or unexpectedly insightful. The Cult Mechanicus' raison d'etre is the realization that religion persists across time and space scales that knowledge alone does not. Thus, by making a religion of knowledge you better guarantee its preserv…

Unfortunately, I think Vernor Vinge scooped you any way. One of the main characters of A Deepness in the Sky was something akin to a software archaeologist (I swear that exact phrase was used, but it’s been a minute) and that book was published in 1999.

Fair, but I didn't get to it until ~2010. And yeah, Vinge uses the exact phrasing https://akkartik.name/post/deepness

Re: Where the goblins came from

#519

Post by IAmBroom »

jagged-chisel wrote: post #387
lukan wrote: post #375

Earlier quoted context omitted.

Well, we did build airplanes out of steel, but there are better (lighter) materials avaiable. But the developement of car engines did directly enabled airplane engines. Not sure if this is the right analogy path, but I kind of suspect similar with LLM's/transformers. They will be a important part.

An important stepping stone, perhaps. But I don’t think the final AGI thing will necessarily contain LLMs.

Perhaps Douglas-Adamsesque the LLMs will specify the AGI.

Re: Where the goblins came from

#520

Post by aldebaran1 »

surgical_fire wrote: post #499
wanderingmind wrote: post #362

Earlier quoted context omitted.

Humanity has been using steel for over a millenia, however it's only in the past 100 years or so we have a good understanding of how carbon interacts with iron at an atomic level to create the strength characteristics that makes it useful. Based on this argument, we should not have used steel, until we had a complete first principles understanding.

This is a very low-effort argument. Humans could understand properties of steel long before they knew how Carbon interacted with Iron. Steel always behaved in a predictable, reproducible way. Empirical experiments with steel usage yielded outputs that could be documented and passed along. You could measure steel for its quality, etc. The same cannot be said of LLMs. This is not to say they are not useful, this was ne…

[dead]
Post reply on HN