Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training for one trillion parameter model backed by Intel and US govt has begun

techradar.com

Post reply on HN
Print view View original

121–130 of 267 posts

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#121

Post by SkyMarshal »

jackschultz wrote: post #118
theropost wrote: post #58

Earlier quoted context omitted.

Ideally no one person or entity controls such a thing. But, would I rather have a Government, or a corporation control AGI? If I had to pick one of two evils, the Government would be the lesser of the two.

For everyone who responds to this question, thinking they're right, please look back and try to find the reason or reasons as to why you pick a government vs corporation. You'll realize you yourself are not making this decision. The number of news stories that have been pushed in front of you, or talking points from people you value as smart or worthy of listening to, all fabricate your reactions and who you consider…

Yeah, whoever controls the information environment controls peoples’ minds. That said, some of us are aware of this dynamic and try to overcome it by reasoning from first principles, eliminating logical fallacies and biases from our training data set, and other techniques.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#122

Post by Pat_Murph »

sixQuarks wrote: post #59
theropost wrote: post #58

Earlier quoted context omitted.

Ideally no one person or entity controls such a thing. But, would I rather have a Government, or a corporation control AGI? If I had to pick one of two evils, the Government would be the lesser of the two.

Has a corporation ever tried to commit genocide?

There are initiatives to have international criminal courts to recognise that corporations are responsable for participating in genocides.

They are not part of the international coûts mandaté at the moment so they are used to shield people behind them.

Here's a paper exploring this possibility

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2151510

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#123

Post by Spooky23 »

postmodest wrote: post #116
munksbeer wrote: post #99

Earlier quoted context omitted.

The opposite for me. Corporation have naked greed as their driving motivation, but that usually doesn't involve killing off all of their customers. That would be quite unprofitable. People elected to government often seem to seek power for powers sake and I have less faith that they'll not harm us.

I'm sure Peter Thiel has no power motive!

Oh no. He's a simple man, guided by the invisible hand of the market. ;)

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#124

Post by Mountain_Skies »

SkyMarshal wrote: post #119
theropost wrote: post #58

Earlier quoted context omitted.

Ideally no one person or entity controls such a thing. But, would I rather have a Government, or a corporation control AGI? If I had to pick one of two evils, the Government would be the lesser of the two.

Depends on the government. If it’s one that’s reasonably answerable to the public, then that may be preferable to a multinational corp. But if it’s an inherently authoritarian govt with a track record of human rights abuse and oppression, then the corp is probably the lesser evil.

The scary fun part about representative democracy is that all of that can change rapidly, and not necessarily in the direction one would want for any given situation. "Throw the bums out" might scare some politicians into slightly better behavior but most just throw more donor money at the problem and it goes away. If not, often an even worse "bum" ends up taking their place. Of course, with the right model behind them, the bums might find a way to manipulate the public consensus enough to never get thrown out.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#125

Post by Footnote7341 »

It will be interesting to see what the government can do here. Can they use their powers to get their hands on the most data?

im still skeptical because new techniques are going to give an order of magnitude efficiency boost to transformer models, so 'just waiting' seems like the best approach for now. I dont think they will be able to just skip to the finish line by having the most money.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#126

Post by tomohelix »

mdemare wrote: post #109
Majromax wrote: post #41

Earlier quoted context omitted.

> I bet if you add up that data it's a lot. Let's Fermi estimate that. A 4k video stream is about 50 megabits/second. Let's say that humans have the equivalent of two of those going during waking hours, one for vision and one for everything else. Humans are awake for 18 hours/day, and we'll say a human's training is 'complete' at 25. Multiply that together, and you end up with 1.8e17 bytes, or 180 petabytes of data.…

Children are awake 12 hours per day, a 90 minute movie used to fit on a CD-ROM, 8 700MB CD-ROMs times 365 is 2TB per year. Alexander the Great became king aged 20, he seemed trained enough by then, so that's 400TB, 4e14 bytes of data. Pretty unconvinced by this argument, it mostly hinges on the video stream size.

I think it is an estimate. Yours is on the extreme low end of things. The GP was using 4K as a standin for touch, smell, taste, pain, hunger, pleasure, etc. Those are a bit more complex than just videos and you can't encode those on a CD.

Regardless, yours still came out to be an order of magnitude higher than what GPT was trained on. So I guess the original argument makes some sense.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#127

Post by immmmmm »

superhumanuser wrote: post #94
theropost wrote: post #58

Earlier quoted context omitted.

Ideally no one person or entity controls such a thing. But, would I rather have a Government, or a corporation control AGI? If I had to pick one of two evils, the Government would be the lesser of the two.

Governments have killed more people than any other institution. Corporations have largely made our lives better.

Oh you should go say that to farmers close to the equator whose crops are failing due to climate change.

All that while fossil fuel companies receive 7000B$ of annual subsidies, and spend billions yearly to fuel disinformation.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#128

Post by tyingq »

Footnote7341 wrote: post #125

It will be interesting to see what the government can do here. Can they use their powers to get their hands on the most data? im still skeptical because new techniques are going to give an order of magnitude efficiency boost to transformer models, so 'just waiting' seems like the best approach for now. I dont think they will be able to just skip to the finish line by having the most money.

If not "the most" data, they may have the most access to data that's exclusively available to them.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#129

Post by Mountain_Skies »

_heimdall wrote: post #86
theropost wrote: post #58

Earlier quoted context omitted.

Ideally no one person or entity controls such a thing. But, would I rather have a Government, or a corporation control AGI? If I had to pick one of two evils, the Government would be the lesser of the two.

It's a logical trap to assume we have to pick who develops an AGI rather than deciding if we want one developed at all. Even in that scenario, I'm not really sure what the distinction would end up being. At least in the US, our government and corporations are so tightly coupled at this point that we're really toeing the waters of Fascism. If a corporation develops an AGI our government can have access to it whenever…

In isolation, perhaps we could choose to not develop such large models, but the globe shrinks every day. If one country declines to create such a beast, other countries still will create their own beast. From what I've seen so far, it's mostly a brute force issue now, not a specialized knowledge issue. Perhaps if all the rumors spilling out of OpenAI about Q* are true, then other countries can be blocked due to lack of the required knowledge, but I have doubts that such secrets would be secret for long.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#130

Post by Spooky23 »

madsbuch wrote: post #96
_heimdall wrote: post #44

I'm sure the government's mission is also to develop an AGI that benefits us all.

This is a particular interesting part of American culture: The sentiment is that it is problematic when the government develops these technologies, but it is completely OK to let private entities develop them.

When the United State deregulated media to essentially eliminate the public service component of media, we created a race to the bottom that we're now realizing in the public space.

The generation of people in their 30s and 40s running the country spent summers listening to Rush Limbaugh over lunch with grandpa, grew up to consume news and comment from message boards and later social media, entered adulthood in the middle of an endless cycle of warfare, and came into their own in this era of propaganda and information warfare. Mainline protestantism has been displaced by Pastor Bob and prosperity gospel. We're replacing the craven boomers with nihlists.

The underlying trope of most propagandistic media is always to discredit the most powerful entity in the room - the government. Why? The money people behind faux-grassroots media are almost always reactionaries who often build their empires on the government teat and don't appreciate the irony. Particularly true of extractive capitalists in the petroleum and materials industries. The US Government giveth, and is the only entity that can taketh. So "rule of law" for me, and "freedumb" for thee.

Post reply on HN