Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On the dangers of stochastic parrots: Can language models be too big? (2021)

dl.acm.org

Post reply on HN
Print view View original

51–60 of 111 posts

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#51

Post by JPLeRouzic »

2bitencryption wrote: post #44

Pure speculation ahead- The other day on Hacker News, there was that article about how scientists could not tell GPT-generated paper abstracts from real ones. Which makes me think- abstracts for scientific papers are high-effort. The corpus of scientific abstracts would understandably have a low count of "garbage" compared to, say, Twitter posts or random blogs. That's not to say that all scientific abstracts are ama…

> "The corpus of scientific abstracts would understandably have a low count of "garbage" compared to, say, Twitter posts or random blogs"

That's certainly true, but it's not by a so large margin, at least in biology.

For example in ALS (a neurodegenerative disease) there is a real breakthrough perhaps every two years, but most papers about ALS (thousands every year) look like they describe something very important.

Similarly for ALZforum the most recent "milestone" paper about Alzheimer disease was in 2012, yet in 2022 alone there were more than 16K papers!

So the ratio signal on noise is close to zero.

https://www.alzforum.org/papers?type%5Bmilestone%5D=mileston...

https://pubmed.ncbi.nlm.nih.gov/?term=alzheimer%27s+disease&...

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#52

Post by ShamelessC »

andrepd wrote: post #47
sthatipamala wrote: post #12

This paper is the product of a failed model of AI safety, in which dedicated safety advocates act as a public ombudsman with an adversarial relationship with their employer. It's baffling to me why anyone thought that would be sustainable. Compare this to something like RLHF[0] which has acheived far more for aligning models toward being polite and non-evil. (This is the technique that helps ChatGPT decline to answer…

Isn't RLHF trivially easy to defeat (as it stands now)?

Assuming a motivated “attacker”, yes. The average user will have no such notion of “jailbreaks”, and it’s at least clear when one _is_ attempting to “jailbreak” a model (given a full log of the conversation and a competent human investigator).

I think the class of problems that remain are basically outliers that are misaligned and don’t trip up the model’s detection mechanism. Given the nature of language and culture (not to mention that they both change over time), I imagine there are a lot of these. I don’t have any examples (and I don’t think yelling “time’s up” when such outliers are found is at all helpful).

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#53

Post by [deleted] »

2bitencryption wrote: post #44

Pure speculation ahead- The other day on Hacker News, there was that article about how scientists could not tell GPT-generated paper abstracts from real ones. Which makes me think- abstracts for scientific papers are high-effort. The corpus of scientific abstracts would understandably have a low count of "garbage" compared to, say, Twitter posts or random blogs. That's not to say that all scientific abstracts are ama…

[deleted]

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#54

Post by ncraig »

2bitencryption wrote: post #44

Pure speculation ahead- The other day on Hacker News, there was that article about how scientists could not tell GPT-generated paper abstracts from real ones. Which makes me think- abstracts for scientific papers are high-effort. The corpus of scientific abstracts would understandably have a low count of "garbage" compared to, say, Twitter posts or random blogs. That's not to say that all scientific abstracts are ama…

Some might say that abstracts are the original clickbait.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#55

Post by didntreadarticl »

monkaiju wrote: post #7

The problems with LLM are numerous but whats really wild to me is that even as they get better at fairly trivial tasks the advertising gets more and more out of hand. These machine dont think, and they dont understand, but people like the CEO of OpenAI allude to them doing just that, obviously so the hype can make them money.

I dont think you understand mate

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#56

Post by visarga »

generalizations wrote: post #38
lumost wrote: post #32

Earlier quoted context omitted.

This is going to happen alot over the next few years. One can fine tune GPT-2 medium on an RTX2070. Training GPT-2 medium from scratch can be done for $162 on vast.ai. The newer H100/Trainium/Tensorcore chips will bring the price down even further. I suspect if one wanted to fully replicate ChatGPT from scratch it would take ~1-2 million including label acquisition. You probably only require ~200-500k in compute. The…

These things have reached the tipping point where they provide significant utility to a significant portion of the computer scientists working on making these things. Could be that the coming iterations of these new tools will make it increasingly easy to write the code for the next iterations of these tools. I wonder if this is the first rumblings of the singularity.

chatGPT being able to write OpenAI API code is great, and all companies should prepare samples so future models can correctly interface with their systems.

But what will be needed is to create an AI that implements scientific papers. About 30% of papers have code implementation. That's a sizeable dataset to train a Codex model on.

You can have AI generating papers, and AI implementing papers, then learning to predict experimental results. This is how you bootstrap a self improving AI.

It does not learn only how to recreate itself, it learns how to solve all problems at the same time. A data engineering approach to AI: search and learn / solve and learn / evolve and learn.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#57

Post by Dylan16807 »

cactusplant7374 wrote: post #43
sthatipamala wrote: post #42

Earlier quoted context omitted.

I am curious to see that trolley problem screenshot. I saw another screenshot where ChatGPT was coaxed into justifying gender pay differences by prompting it to generate hypothetical CSV or JSON data. Basically you have to convince modern models to say bad stuff using clever hacks (compared to GPT-2 or even early GPT-3 where it would just spout straight-up hatred with the lightest touch). That's very good progress an…

> I saw another screenshot where ChatGPT was coaxed into justifying gender pay differences by prompting it to generate hypothetical CSV or JSON data. I remember seeing that on Twitter. My impression was author instructed the AI to discriminate by gender.

Did the author tell it which way or by how much?

If I say to discriminate on some feature and it consistently does it the same way, that's still a pretty bad bias. It probably shows up in other ways.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#58

Post by Dylan16807 »

LarryMullins wrote: post #46
monkaiju wrote: post #7

The problems with LLM are numerous but whats really wild to me is that even as they get better at fairly trivial tasks the advertising gets more and more out of hand. These machine dont think, and they dont understand, but people like the CEO of OpenAI allude to them doing just that, obviously so the hype can make them money.

> These machine dont think And submarines don't swim.

And it would be bad for a submarine salesman to go to people that think swimming is very special and try to get them believing that submarines do swim.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#59

Post by zzzeek »

srvmshr wrote: post #28
visarga wrote: post #21

Earlier quoted context omitted.

It was activism masquerading as science. Many researches noted that positives and negatives were not presented in a balanced way. New approaches and efforts were not credited.

I haven't kept track but the activism of the trio could be severe sometimes. (Anecdotally, I have faced a bite-sized brunt: When discussion surrounding this paper was going on in Twitter, I had mentioned in my timeline (in a neutral tone) that "dust needed to settle to understand what was going wrong". This was unfortunately picked up & RTed by Gebru & the mob responded by name-calling, threatening DMs accusing me of…

> This was unfortunately picked up & RTed by Gebru & the mob responded by name-calling, threatening DMs accusing me of racism/misogyny etc, and one instance of a call to my employer asking to terminate me - all for that one single tweet.

Wait until an LLM flags your speech and gets you in trouble. That'll be a real hoot compared to random individuals who likely have been chased off Twitter by now.

Re: On the dangers of stochastic parrots: Can language models be too big? (2021)

#60

Post by LarryMullins »

Dylan16807 wrote: post #58
LarryMullins wrote: post #46

Earlier quoted context omitted.

> These machine dont think And submarines don't swim.

And it would be bad for a submarine salesman to go to people that think swimming is very special and try to get them believing that submarines do swim.

Why would that be bad? A submarine salesman convincing you that his submarine "swims" doesn't change the set of missions a submarine might be suitable for. It makes no practical difference. There's no point where you get the submarine and it meets all the advertised specs, does everything you needed a submarine for, but you're unsatisfied with it anyway because you now realize that the word "swim" is reserved for living creatures.

And more to the point, nobody believes that "it thinks" is sufficient qualification for a job when hiring a human, so why would it be different when buying a machine? Whether or not the machine "thinks" doesn't address the question of whether or not the machine is capable of doing the jobs you want it to do. Anybody who neglects to evaluate the functional capability of the machine is simply a fool.

Post reply on HN