Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

181–190 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#181

Earlier quoted context omitted.

Like Facebook, Twitter and every other forum did? I wonder if there were any consequences if they didn't.

Basically they've taken a step further with censorship of all media not controlled by government which at the time (2014) couldn't been penalized whatsoever.

And what if they hadn't?

Re: YaLM-100B: Pretrained language model with 100B parameters

#182

Earlier quoted context omitted.

> I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Isn’t that already the case? Sure, it costs $60K, but that is accessible to a surprisingly large minority, considering the potency of this software.

Nitpick: This uses 8x A100 which are at least $10k a piece to my knowledge. Add in the computer and you're closer to $100k.

I believe you're confusing the amount of A100 graphics cards used to train the model (the cluster was actually made up of 800 A100s), and the amount you need to run the model :

> The model [...] is supposed to run on multiple GPUs with tensor parallelism.

> It was tested on 4 (A100 80g) and 8 (V100 32g) GPUs, [but should work] with ≈200GB of GPU memory.

I don't know what the price of a V100 is, but given $10k a piece for A100s we would be closer to the $60k estimate.

Re: YaLM-100B: Pretrained language model with 100B parameters

#183
post #74
post #60

Earlier quoted context omitted.

...what? 60 thousand dollars for a dedicated computer that you can't use is not everyone, not on their own computers, and is also a crazy large amount of money for nearly everyone. Sure there are some that could, but that's not what I said.

Indeed. What "everyone" can use is a ~$200 smartphone, so there's a ~300x gap to be bridged.

Most of the cost of a phone isn’t the processor, so probably closer to x1000. Hardware may get that much cheaper, but it was never guaranteed, and we’re not making progress as fast as we used to.

Re: YaLM-100B: Pretrained language model with 100B parameters

#184
post #45

Earlier quoted context omitted.

Well... I'm sorry if I reach for the reductio at Hitlerum, but any achievements Nazi scientists might have reached in concentration camps are definitely tainted. Similarly, achievements in the field of online consumer analysis in a country where consumer-privacy protections are nonexistent, surely should be considered tainted...?

And Yandex's AI work got helped by the Russian invasion of Ukraine how, exactly? Did they train the bots on Ukrainian captives first?

Its developers stayed complicit with a company with a biggest propagandist pro-war resource in a country (Yandex News).

Re: YaLM-100B: Pretrained language model with 100B parameters

#185
post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

That's definitely the future, personalized entertainment and social interactions will be big. I could watch a movie made for me, and discuss it with a bunch of chat bots. The future will be bubbly as hell, people will be decaying in their safe places as the hellscape rages on outside.

> I could watch a movie made for me

We're a long, long way from this. Stringing words/images together into a coherent sequence is arguably the easy bit of creating novels/films, and computers still lag a long way behind humans in this regard.

Structuring a narrative is a harder, subtler step. Our most advanced ML solutions are improving rapidly, but often struggle with coherence over a single paragraph; they're not going to be doing satisfying foreshadowing and emotional beats for a while.

Re: YaLM-100B: Pretrained language model with 100B parameters

#186
post #157

Earlier quoted context omitted.

That's definitely the future, personalized entertainment and social interactions will be big. I could watch a movie made for me, and discuss it with a bunch of chat bots. The future will be bubbly as hell, people will be decaying in their safe places as the hellscape rages on outside.

You really dont want to live in Mindwarp (1992 Bruce Campbell movie) or in this !114! year old short story https://en.wikipedia.org/wiki/The_Machine_Stops

The Machine Stops is eerily prescient - or perhaps just keenly observant of trends visible even at the time - but in fairness the humans in it are not socially isolated, as such; they do not converse with bots, but rather with each other. The primary social activity in the The Machine Stops is the Zoom meeting.

I do not look forward to the day when that story becomes an optimistic view of the future.

Re: YaLM-100B: Pretrained language model with 100B parameters

#187
post #72

Earlier quoted context omitted.

Underrated comment, considering the history of NATO expansion, color revolutions and the hundreds of thousands killed in pursuit of reckless ideological overt or covert warfare.

Yikes

Unsure what this comment means.

Re: YaLM-100B: Pretrained language model with 100B parameters

#188
post #163

Earlier quoted context omitted.

What have Israel connections got to do with shadiness? What are you insinuating?

Settling land that was recently taken from Palestinian families by force, often (literally) knocking the existing houses over with a bulldozer. In my opinion, that's a moral red flag to participate in such an atrocity.

What has Yandex got to do with any of that? Is everyone who lives in a country that commits atrocities "shady as fuck"?

: reading closely I see that the initial allegation did use the word "settlement", and indeed that would constitute ethically questionable behavior. However, a sibling comment refutes this.

Re: YaLM-100B: Pretrained language model with 100B parameters

#189
post #24

Earlier quoted context omitted.

the age old question if the art should be linked or disconnected from the artist

This is nothing like that, because the question is not one of their own personal actions - but of their nationality or ethnicity. That, until about 4 months ago, would have been widely acknowledged as racism. The difference between holding values, and holding values when convenient rather sums up the entirety of human history in one phrase.

Yeah, it's certainly about ethnicity, not at all about being controlled by a government which is in the process of perpetrating genocide.

Re: YaLM-100B: Pretrained language model with 100B parameters

#190
post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

Yes, the vision is that everyone has an AI cube in their house.

Then, we'll hack all those cubes to build an AGI.
Post reply on HN