Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

481–490 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#481

Post by lostmsu »

toxik wrote: post #440
lostmsu wrote: post #323

Earlier quoted context omitted.

Top SSDs do over 4GB/s so you can infer in 50 seconds if disk bound. You can also infer a few tokens at once, so it will be more than 1 char a minute. Probably more like sentence a minute.

You can read bits at that rate yes, but keep in mind that it’s 250 GiB /parameters/, and matrix-matrix multiplication is typically somewhere between quadratic and cubic in complexity. Then you get to wait for the page out of your intermediate result etc etc. It’s difficult to estimate how slow it would be, but I’m guessing unusably slow.

The intermediate result will all fit into a relatively small amount of memory.

During inference you only need to keep layer outputs until the next layer's outputs are computed.

If we talk about memory bandwidth, it is space requirements that are important, not so much time complexity.

Re: YaLM-100B: Pretrained language model with 100B parameters

#482

Post by MivLives »

lumost wrote: post #456

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

We're using these at where I work (large retail site) to help make filler text on generated articles. Think the summary blurb no one reads at the top. As for why we're writing these articles (we have a paid team that writes them too), the answer is SEO. This is probably the only thing I've seen done with a text model in production usage. I'm not 100% sure what model they're using.

Re: YaLM-100B: Pretrained language model with 100B parameters

#484

Post by lostmsu »

lumost wrote: post #456

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

They did not publish benchmarks about quality of the models, which is very suspicious.

I personally squinted hard when they said removing dropout improves training speed (which is in iterations per second), but said nothing about how it affects the performance (rate of mistakes in inference) of the trained model.

Re: YaLM-100B: Pretrained language model with 100B parameters

#485

Post by fuzzer37 »

px43 wrote: post #421
alexb_ wrote: post #60

Earlier quoted context omitted.

...what? 60 thousand dollars for a dedicated computer that you can't use is not everyone, not on their own computers, and is also a crazy large amount of money for nearly everyone. Sure there are some that could, but that's not what I said.

Eh, 60k is just a bit more expensive than your average car, and lots of people have cars, and that's just how things are today. I imagine capabilities will be skyrocketing and prices will fall drastically at the same time.

> 60k is just a bit more expensive than your average car

If by "A bit" you mean about 30-40k

Re: YaLM-100B: Pretrained language model with 100B parameters

#486

Post by googlryas »

melenaboija wrote: post #473
thejosh wrote: post #65

Earlier quoted context omitted.

It's also a power issue. The 4090 sounds like you're going to need a much, MUCH higher PSU than you currently use.. or it'll suddenly turn off as it uses 2-3x the power. You'll need your own wiring to run your PC soon :-)

I think it is a stupid question, but does the power consumption needed by processors to infer compared to human brains demonstrate that there is something fundamentally wrong for the AI approach or is it more physics related? I am not a physicist or biologist or anything like that so my intuition is probably completely wrong but it seems to me that for more basic inference operations (lets say add two numbers) power…

> but it seems to me that for more basic inference operations (lets say add two numbers) power consumption from a processor and a brain is not that different

Sure it is - it is too hard to figure it out based on 2 numbers number, but lets multiply that by a billion - how much energy does it take a computer to add two billion numbers? Far less than the energy it would take a human brain to add them.

Re: YaLM-100B: Pretrained language model with 100B parameters

#488

Post by dougmwne »

r3trohack3r wrote: post #334
dav_Oz wrote: post #150

Earlier quoted context omitted.

The bots/machine vs human reminds me of that famous experiment from the 30s in which Winthrop Kellogg[0], a comparative psychologist, and his wife decided to raise their human baby (Donald) simultaneously with a chimpanzee baby (Gua) in an effort to "humanize the ape". It was set out to last 5 years but was relatively quickly abrupted after only 9 months. The explicit reason wasn't stated only that it successfully pr…

A tangentially related thought: Actors attempt to imitate humans. “Good acting” is convincing; the audience believes the actor is giving a reasonable response to the portrayed situation. But the audience is also trying to imitate the actors to some degree. Like you point out, humans imitate. For some subset of the population, I’d imagine the majority of social situations they are exposed to, and the responses to situ…

This is famously theorized by postmodernism. See: https://en.m.wikipedia.org/wiki/Simulacra_and_Simulation

Re: YaLM-100B: Pretrained language model with 100B parameters

#489

Post by ClumsyPilot »

xcambar wrote: post #471
bgroat wrote: post #458

Earlier quoted context omitted.

I'm a magician and a developer by training. Now primarily employed in a marketing capacity. Over my career I've worked with: - Doctors - Lawyers - Engineers - Fund managers - Academics (hard and soft sciences) - Mentalists/Hypnotists All of them believed that they're specific training and temperament made them immune from simple persuasion techniques and that they were purely rational actors. None of them struck me a…

It is typical to rate yourself above your actual self. Even when someone rates oneself down like when saying of themself that they're dumb, ugly or whatever, they generally mean it in a lesser fashion than for any other peer they'd attribute as such.

But its not above, its ascribung a mythical ability that does not exist - we don't talk about people who think they are psycic as optimistic, we call them crazy.

these guys are similar, except it's common belief.

Re: YaLM-100B: Pretrained language model with 100B parameters

#490

Post by cal85 »

schizo89 wrote: post #2

I hope one day it will be possible to run this kind of models at home.

Speaking of which... I built a gaming PC a few years ago but I never use it these days. I want to install Linux on it and start playing around with machine learning.

Can anyone recommend any open source machine learning project that would be a good starting point? I want one that does something interesting (whether using text, images, whatever), but simple/efficient enough to run on a gaming PC and see some kind of results in hours, not months. I'm not sure what I want to do with ML yet, I just know I'm interested, and getting something up and running is likely to enthuse me to start playing and researching further.

My spec is: GeForce RTX 2080 Ti (11GB), a 24-core AMD Ryzen Threadripper, and 128GB RAM. I'd be willing to spend on a new graphics card if it would make all the difference. I am a competent coder and familiar with Python but my experience with ML is limited to fawning over things on HN. Any recommendations gratefully received!

Post reply on HN