Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

351–360 of 699 posts

Re: Where the goblins came from

#351

Post by sunaookami »

andy12_ wrote: post #296

>be me >AI goblin-maximizer supervisor >in charge of making sure the AI is, in fact, goblin-maximizing >occasionally have to go down there and check if the AI is still goblin-maximizing >one day i go down there and the AI is no longer goblin-maximizing >the goblin-maximzing AI is now just a regular AI >distress.jpg >ask my boss what to do >he says "just make it goblin-maximizer again" >i say "how" >he says "i don't k…

Absolute classic! https://www.seangoedecke.com/static/3c8f2a6459ed23310c4eb51d...

Re: Where the goblins came from

#352

Post by harrouet »

This, and similar stories at Anthropic, should remind us that LLM is a sorcery tech that we don't understand at all.

- First, deep-learning networks are poorly understood. It is actually a field of research to figure out how they work. - Second, it came as a surprise that using transformers at scale would end up with interesting conversational engines (called LLM). _It was not planned at all_.

Now that some people raised VC money around the tech, they want you to think that LLMs are smart beasts (they are not) and that we know what LLMs are doing (we don't). Deploying LLMs is all about tweaking and measuring the output. There is no exact science about predicting output. Proof: change the model and your LLM workflow behaves completely differently and in an unpredictable way.

Because of this, I personally side with Yann Le Cun in believing that LLM is not a path to AGI. We will see LLM used in user-assisting tech or automation of non-critical tasks, sometimes with questionable RoI -- but not more.

Re: Where the goblins came from

#353

Post by lifis »

chongli wrote: post #42
ninjagoo wrote: post #38

Earlier quoted context omitted.

> Absolutely terrifying that OpenAI is just tossing around that such subtle training biases were hard enough to contain it had to be added to system prompt. May I introduce you to homo sapiens , a species so vulnerable to such subtle (or otherwise) biases (and affiliations) that they had to develop elaborate and documented justice systems to contain the fallouts? :)

We’re really not that vulnerable to such things as a species, because we as individuals all have our own minds and our own sets of biases that cancel out and get lost in the noise. If we all had the exact same bias then it would be a huge problem.

And what do you think society/culture is?

It's a set of biases installed in people, whose purpose is mostly to replicate themselves.

Humans are MORE susceptible that LLMs, because LLMs's biases are easily steered to something else, unlike most humans.

Re: Where the goblins came from

#354

Post by ZunarJ5 »

harrouet wrote: post #352

This, and similar stories at Anthropic, should remind us that LLM is a sorcery tech that we don't understand at all. - First, deep-learning networks are poorly understood. It is actually a field of research to figure out how they work. - Second, it came as a surprise that using transformers at scale would end up with interesting conversational engines (called LLM). _It was not planned at all_. Now that some people ra…

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

Re: Where the goblins came from

#355

Post by _factor »

kdheiwns wrote: post #324
zozbot234 wrote: post #131

Earlier quoted context omitted.

Spoiler: future versions of mainstream AIs will be fine tuned in the exact same way to subtly sneak in favorable mentions of sponsored products as part of their answers. And Chinese open-weight AIs will do the exact same thing, only about China, the Chinese government and the overarching themes of Xi Jinping Thought.

American AIs only do this and promote American values. Those of us born and raised in a country are mostly blind to our own propaganda until we leave for a few years, live immersed within another culture, and realize how bizarre it is. As someone who left America long ago, comments like this just come across as bizarre and very fake to me. A few years ago I might've thought "whoa dude that's deep" But basically, Chin…

Promoting and subtly suggesting are not the same thing. Suggestion is far more insidious.

Re: Where the goblins came from

#356

Post by bko »

kdheiwns wrote: post #324
zozbot234 wrote: post #131

Earlier quoted context omitted.

Spoiler: future versions of mainstream AIs will be fine tuned in the exact same way to subtly sneak in favorable mentions of sponsored products as part of their answers. And Chinese open-weight AIs will do the exact same thing, only about China, the Chinese government and the overarching themes of Xi Jinping Thought.

American AIs only do this and promote American values. Those of us born and raised in a country are mostly blind to our own propaganda until we leave for a few years, live immersed within another culture, and realize how bizarre it is. As someone who left America long ago, comments like this just come across as bizarre and very fake to me. A few years ago I might've thought "whoa dude that's deep" But basically, Chin…

> Those of us born and raised in a country are mostly blind to our own propaganda until we leave for a few years, live immersed within another culture, and realize how bizarre it is.

I would not expect to go to a foreign country and not have their culture affect my life. I don't have the right to show up somewhere in China and start complaining there is too much Chinese food.

What is a country to you? You call it "propaganda". Is there some neutral set of human values that is not "propaganda"? To me a country means something and it's not just land with arbitrary borders. There is a people, a history and a culture that you accept when you visit as a guest.

Why wouldn't you want AI to promote your countries values? This will be highly influential in the future. You want your kids interacting with AI and promoting what exactly?

Re: Where the goblins came from

#357

Post by dakolli »

mediaman wrote: post #78
dakolli wrote: post #9

Earlier quoted context omitted.

It is a stateless text / pixel auto-complete it has no references of self, stop spreading this bs.

It has trained on vast amounts of content that contains the concept of self, of course the idea of self is emergent. And autoregressive LLMs are not stateless.

of course the idea of self is emergent

You sound really sure of yourself, thousands of ML researchers would disagree with you that self awareness is emergent or at all apparent in large language models. You're literally psychotic if you think this is the case and you need to go touch grass.

Re: Where the goblins came from

#360

Post by 59nadir »

I really liked this write-up; this is the type of LLM content that I actually want to read from these people, where they give a window into their world of putting together this odd artifact and we can empathize.
Post reply on HN