Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training for one trillion parameter model backed by Intel and US govt has begun

techradar.com

Post reply on HN
Print view View original

261–267 of 267 posts

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#261

Post by _heimdall »

jncfhnb wrote: post #258
_heimdall wrote: post #246

Earlier quoted context omitted.

It sounds like you're describing coordinated action as both the cause of and solution to the same problem. Stopping this accumulation of power takes very little, its undoing the accumulation of power that is costly.

Stopping accumulation of power is very hard when you have no accumulated power. I’m not sure how you could possibly believe otherwise unless you’re one of those believers in magical harmonious anarchy

When there isn't accumulated power you only have to avoid giving up power and sovereignty. There's nothing else to stop.

I don't consider myself an anarchist, partly be cause the term has been repurposed so many times now that it doesn't have a clear meaning.

The belief in magical harmony is unrealistic regardless of the model that's supposed to create it, whether its anarchism or federalism.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#262

Post by spott »

bane wrote: post #201
awongh wrote: post #180

Earlier quoted context omitted.

How are the optimal architectures for weather simulations and LLM training different?

tbh, I'm not entirely informed on what the requirements are for LLM training, but I've noticed that nearly all of the teams don't use unified memory (what makes a super computer "super"). I believe I read somewhere openAI uses a K8s cluster[1] and other teams I know seem to work with other similar non-unified memory systems. If there's no advantage, part of what makes supercomputers expensive is this memory interconn…

What is this “unified memory” you speak of?

Modern supercomputers are just a bunch of computers with a (very) high speed interconnect. As far as I’m aware, pretty much every multi node job on a supercomputer uses MPI to communicate, and there isn’t any sort of “unified memory” system.

It has been a few years since I worked on one of the big supercomputers though, so maybe you have better knowledge than me. I’d love more information.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#263

Post by jncfhnb »

_heimdall wrote: post #261
jncfhnb wrote: post #258

Earlier quoted context omitted.

Stopping accumulation of power is very hard when you have no accumulated power. I’m not sure how you could possibly believe otherwise unless you’re one of those believers in magical harmonious anarchy

When there isn't accumulated power you only have to avoid giving up power and sovereignty. There's nothing else to stop. I don't consider myself an anarchist, partly be cause the term has been repurposed so many times now that it doesn't have a clear meaning. The belief in magical harmony is unrealistic regardless of the model that's supposed to create it, whether its anarchism or federalism.

So when you have $10 and your neighbor has $200 in whatever assets or currency your little community needs, and your neighbor decides to invest it, you’re gonna what, attack him? Gather your other neighbors and demand he share?

Uh oh, what if he spent $50 buying muscle?

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#264

Post by nyc_data_geek1 »

astrange wrote: post #222
nyc_data_geek1 wrote: post #216

Earlier quoted context omitted.

Ask the people in Bhopal how DOW Chemical has made their lives better, or the workers in Shenzhen Apple factories, or the people making your clothes in the third world, or the people picking through trash mountains for a living, how much corporations have made their lives better.

The people working at Foxconn are doing it voluntarily because it pays better than their other alternatives. Same for Bangladesh clothing factories.

There is voluntary, and then there is voluntary.

https://www.washingtonpost.com/technology/2019/09/09/apple-a...

Why don't you ask the people who were slaving in United Fruit plantations how much corporations improved their lives? Or East India Company colony subjects?

Corporations benefit those who benefit from them, often at the expense of everyone else/extraction of the commons.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#265

Post by nyc_data_geek1 »

astrange wrote: post #220
AndrewKemendo wrote: post #143

Earlier quoted context omitted.

Precisely this Link here too for everyone: https://archive.org/details/WarIsARacket

No, this is wrong. Saying war is for money gives war too much credit - it's not good for anything and it doesn't make anyone any money. The opportunity cost is too great. The US does not care about "the petrodollar". Nor do we care about foreign oil. Nor do we have remotely the same economy we did in 1935. In fact, war is not even profitable for our military industrial complex. They profit from the threat of war beca…

It's not true, though. War is an effective wealth transfer mechanism - from the lower/working classes to the rich war parasite class. Halliburton absolutely profited from the Iraq invasion, though in aggregate you're absolutely right that prosperity is reduced by wars.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#266

Post by thefounder »

api wrote: post #159
thefounder wrote: post #158

Earlier quoted context omitted.

Yeah but AGI would control both in the end.

Looking at our two leading presidential candidates could that be an improvement? Maybe AGI takes over after it emerges as a solution to our inability to select competent candidates for political office.

>> Looking at our two leading presidential candidates could that be an improvement?

I think it could be better until it won't. Like with dictators sometimes they "do good" but the issue is that at some point they will start doing more harm than good and they may not want to go away.

A succesful revolution against an AGI ruler may become impossible.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#267

Post by Gormo »

astrange wrote: post #223
Gormo wrote: post #191

Earlier quoted context omitted.

Aren't deaths from tobacco directly caused by intentional individual behavior? I'm not sure how that relates to either example.

Individual behavior doesn't really exist, you don't have choices outside the ones provided by society. It's also a bit silly to mention it in the context of a literally physically addictive drug. Interestingly schizophrenic people almost universally smoke, and it's thought this is because it's a better treatment than the actual drugs we give them, though that's not proven.

> Individual behavior doesn't really exist, you don't have choices outside the ones provided by society.

No, that's quite backwards. Individual behavior is effectively the only thing that objectively exists, as "society" is just an emergent pattern of aggregated individual behavior. Individuals' choices are limited by their own particular capacities, and "society" factors into that only in the sense that the choices of other individuals in the same bounded context, in aggregate, can function as environmental constraints. So a more accurate statement is "every individual has full freedom of choice, but external constraints including the behavior of others may limit one's ability to fully execute the choices one has made".

Given that the act of smoking entails deliberately choosing to purchase a product, light it on fire, and actively inhale the fumes -- all behaviors which are necessarily the result of purposeful intention -- and given that within any social context one will find some people who smoke but a great many who do not, along with many who have chosen to quit (despite the external influences and incentives being largely constant), it seems impossible to attribute smoking to anything other than conscious individual choice.

I once asked family member why they smoke, and the response was "because I love it". I had, and could have, no retort to that -- if another person consciously chooses to make the risk/reward tradeoff in favor of short-term pleasure rather than longer-term health, then there's no further argument to be had, and the only demand that any other party could make is that they not be exposed to negative externalities resulting from that choice.

Post reply on HN