Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training for one trillion parameter model backed by Intel and US govt has begun

techradar.com

Post reply on HN
Print view View original

211–220 of 267 posts

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#211

Post by Jensson »

Gormo wrote: post #190
jncfhnb wrote: post #155

Earlier quoted context omitted.

Monopoly is the natural state. Government is the only reason we have any alternatives.

No, it is very much the other way around. Government is itself a monopoly , and has historically been justified by the intention to mitigate the "war of all against all" that emerges from chaotic competition between divergent factions in a raw state of nature. But the modern status quo is so massively skewed toward government that the benefits from mitigating the worst cases of competition are vastly eclipsed by the…

Modern governments are a democratic monopoly. Corporations aren't democratic, if a corporation gets powerful enough to overthrow the government and replace them you will be in much worse hands than if the current democratic government could remain in power.

So we shouldn't cheer on private corporations that developers technology that could allow them to replace the government, that is really really scary if they actually succeeds in doing it.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#212

Post by 2OEH8eoCRo0 »

Jensson wrote: post #209
2OEH8eoCRo0 wrote: post #206

Earlier quoted context omitted.

Hopefully not.

Why not? Would be cool with some new open source models.

I agree but I don't think our adversaries should get a freebie that's trained on our scientific data at a national laboratory.

Which makes me wonder. I'm not sure this applies here but say you train a model on classified information, is the model/weights then classified?

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#213

Post by rkagerer »

Jensson wrote: post #166
cschmidt wrote: post #160

Earlier quoted context omitted.

I just asked GPT-4, and it denies being a MoE model. User: Are you an MoE model? ChatGPT: No, I am not based on a Mixture of Experts (MoE) model. My underlying architecture is based on the GPT (Generative Pre-trained Transformer) framework, specifically the GPT-4 version. This architecture is a large-scale transformer-based neural network, but it does not use the MoE approach. In a GPT model like mine, the entire mod…

That is almost surely a hallucination.

Curious, why do people gravitate to the term "hallucination" (as if it's a person not a tool) instead of just plainly stating that it's wrong?

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#214

Post by _lvbh »

2OEH8eoCRo0 wrote: post #212
Jensson wrote: post #209

Earlier quoted context omitted.

Why not? Would be cool with some new open source models.

I agree but I don't think our adversaries should get a freebie that's trained on our scientific data at a national laboratory. Which makes me wonder. I'm not sure this applies here but say you train a model on classified information, is the model/weights then classified?

That’s like saying we should stop all education in America so that Chinese people don’t come in and steal our knowledge.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#215

Post by therealdrag0 »

rkagerer wrote: post #213
Jensson wrote: post #166

Earlier quoted context omitted.

That is almost surely a hallucination.

Curious, why do people gravitate to the term "hallucination" (as if it's a person not a tool) instead of just plainly stating that it's wrong?

It’s a chat bot so a lot of language will be anthropomorphic. Even still this definition fits well “3. a false belief or impression; illusion; delusion.”

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#216

Post by nyc_data_geek1 »

superhumanuser wrote: post #94
theropost wrote: post #58

Earlier quoted context omitted.

Ideally no one person or entity controls such a thing. But, would I rather have a Government, or a corporation control AGI? If I had to pick one of two evils, the Government would be the lesser of the two.

Governments have killed more people than any other institution. Corporations have largely made our lives better.

Ask the people in Bhopal how DOW Chemical has made their lives better, or the workers in Shenzhen Apple factories, or the people making your clothes in the third world, or the people picking through trash mountains for a living, how much corporations have made their lives better.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#217

Post by Gormo »

Jensson wrote: post #211
Gormo wrote: post #190

Earlier quoted context omitted.

No, it is very much the other way around. Government is itself a monopoly , and has historically been justified by the intention to mitigate the "war of all against all" that emerges from chaotic competition between divergent factions in a raw state of nature. But the modern status quo is so massively skewed toward government that the benefits from mitigating the worst cases of competition are vastly eclipsed by the…

Modern governments are a democratic monopoly. Corporations aren't democratic, if a corporation gets powerful enough to overthrow the government and replace them you will be in much worse hands than if the current democratic government could remain in power. So we shouldn't cheer on private corporations that developers technology that could allow them to replace the government, that is really really scary if they actu…

> Modern governments are a democratic monopoly.

The fact that governments used contrived symbolic rituals to get arbitrary subsets of arbitrary aggregations of people to express nominal approval of their behavior does nothing whatsoever to alter either the empirical nature or the ethical implicatoins of its monopoly.

> Corporations aren't democratic

Good -- this means that it's harder for them to appeal to vague symbolism to convince people that their actions are inherently legitimate, which in turn means that they are under greater scrutiny to justify their actions, each on its own merits.

> if a corporation gets powerful enough to overthrow the government and replace them you will be in much worse hands than if the current democratic government could remain in power.

Large vested interests are already extremely adept at co-opting nominally "democratic" government and using the very monopoly you're trying to justify as a way of obtaining top-down power that they'd never be able to acquire on their own -- they have zero interest in overthrowing anything.

Regulatory capture is the principal mechanism of corruption in the modern world, and it's astonishing to me that people keep arguing for expanding the reach of the regulatory state in order to reduce the dominance of large corporations, when the actual effect is always to amplify it.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#218

Post by astrange »

toddmorey wrote: post #157
munksbeer wrote: post #99

Earlier quoted context omitted.

The opposite for me. Corporation have naked greed as their driving motivation, but that usually doesn't involve killing off all of their customers. That would be quite unprofitable. People elected to government often seem to seek power for powers sake and I have less faith that they'll not harm us.

I sure see a lot of power for power’s sake inside of companies. OpenAI’s stated mission is AGI that can replace half the population in “economically valuable” work. I get that with some creativity you can see that as a net benefit for humanity, but across at least a generation that’s going to be a rough & destabilizing transition.

Creativity doesn't come into it; the idea of jobs being lost is folk economics that no labor economist believes is possible. So all this shows is that they were founded by amateur AI doomers. We already knew that though.

There is one person out there who decides if you have a job and it's not anyone working on AI. It's the chairman of the Federal Reserve.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#219

Post by feoren »

ComputerGuru wrote: post #152
Majromax wrote: post #41

Earlier quoted context omitted.

> I bet if you add up that data it's a lot. Let's Fermi estimate that. A 4k video stream is about 50 megabits/second. Let's say that humans have the equivalent of two of those going during waking hours, one for vision and one for everything else. Humans are awake for 18 hours/day, and we'll say a human's training is 'complete' at 25. Multiply that together, and you end up with 1.8e17 bytes, or 180 petabytes of data.…

This is completely and fundamentally an incorrect approach from start to finish. The human body - and the human mind - do have electrical and logic components but they are absolutely not digital logic. We do not see in “pixels”. The human mind is an analog process. Analog computing is insanely powerful and exponentially more efficient (time, energy, bandwidth) than digital computing but it is ridiculously hard to pac…

You missed the point of the exercise. Of course it's extremely difficult to compare the two, but the question was: do humans get nearly as much training data as LLMs do? This analysis is good enough to say "actually humans receive much more raw input data than LLMs during their 'training period'." You're concerned with what the brain is doing with all that data compared to LLMs, but that's not the point of the exercise.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#220

Post by astrange »

AndrewKemendo wrote: post #143
nyc_data_geek1 wrote: post #141

Earlier quoted context omitted.

War Is A Racket, as Smedley Butler accurately wrote.

Precisely this Link here too for everyone: https://archive.org/details/WarIsARacket

No, this is wrong. Saying war is for money gives war too much credit - it's not good for anything and it doesn't make anyone any money. The opportunity cost is too great.

The US does not care about "the petrodollar". Nor do we care about foreign oil. Nor do we have remotely the same economy we did in 1935.

In fact, war is not even profitable for our military industrial complex. They profit from the threat of war because that funds development of untested superweapons. Actual wars mean all that stuff actually has to work though, and there's a chance you'll get nationalized.

But if it was good for some MIC rich people, it still wouldn't matter, because they're outnumbered by all the other rich people it's bad for.

Post reply on HN