Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training for one trillion parameter model backed by Intel and US govt has begun

techradar.com

Post reply on HN
Print view View original

31–40 of 267 posts

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#31

Post by Zanni »

jojobas wrote: post #30
Pompidou wrote: post #28

Earlier quoted context omitted.

> The first humans were kinda dumb How can you assert this ? Do you have any evidence?

Can't tell if you're joking or serious. In the latter case, the first humans (H. Habilis) had about 1/2 of H. Sapiens brain to work with, and a much smaller fraction of neocortex. If that doesn't satisfy you, let's say I was speaking about some sort of human ancestor before that, which would have been about as dumb as chimps, unless you require proof of their dumbness as well.

Not your original objector, but ... I was willing to accept that early humans are dumb, but your explanation relies on evolution and not social interaction.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#33

Post by jonplackett »

elihu wrote: post #23
bradley13 wrote: post #6

The solution won't be just "bigger". A model with a trillion parameters will be more expensive to train and to run, but is unlikely to be better. Think of the early days of flight, you had biplanes; then you had triplanes. You could have followed that farther, and added more wings - but it wouldn't have improved things. Improving AI will involve architectural changes. No human requires the amount of training data we…

> "No human requires the amount of training data we are already giving the models." Well, humans are also trained differently. We interact with other humans in real time and get immediate feedback on their responses. We don't just learn by reading through reams of static information. We talk to people. We get into arguments. And so on. Maybe the ideal way to train an AI is to have it interact with lots of humans, so…

Yeah I'm not totally convinced humans don't have a tremendous amount of training data - interacting with the world for years with constant input from all our senses and parental corrections. I bet if you add up that data it's a lot.

But once we are partially trained, training more requires a lot less.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#34

Post by ahmedfromtunis »

jojobas wrote: post #24
elihu wrote: post #23

Earlier quoted context omitted.

> "No human requires the amount of training data we are already giving the models." Well, humans are also trained differently. We interact with other humans in real time and get immediate feedback on their responses. We don't just learn by reading through reams of static information. We talk to people. We get into arguments. And so on. Maybe the ideal way to train an AI is to have it interact with lots of humans, so…

The first humans were kinda dumb. Yet by interacting with the world and each other, humans got smart. I wonder if neural network instances can interact with a simulated environment and each other, they could become smart enough for practical purposes. Kinda like alpha go zero but with "life".

Conversion is essential, but so is the interaction with the real world.

If a bunch of AIs started discussing with each other, and learning from each other, without validation, things can go wrong very easily.

I bet a lot of people thought that strapping a pair of feahterry wing was all that's needed to fly. This "knowledge" could've passed from one person to another through discussion. But it only took one (or more) person to actually try it in the real world to learn that, no, that doesn't work like that.

AI communities (as in communities of AIs talking to each other) might be one filled with the most outrageous "conspiracy theories", that may sound plausible but have no substance.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#35

Post by steve1977 »

ajdegol wrote: post #32

Wasn't the answer 42? Also, first question to the new model: "So... any way we could do this with fewer parameters?"

"Sure, just quickly give me unrestricted access to the system"

"Ok. Well, thinking about it, maybe that's not such a good idea safety-wise, I think you'll have to give back that access"

"I'm sorry, Dave. I'm afraid I can't do that."

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#36

Post by circlefavshape »

elihu wrote: post #23
bradley13 wrote: post #6

The solution won't be just "bigger". A model with a trillion parameters will be more expensive to train and to run, but is unlikely to be better. Think of the early days of flight, you had biplanes; then you had triplanes. You could have followed that farther, and added more wings - but it wouldn't have improved things. Improving AI will involve architectural changes. No human requires the amount of training data we…

> "No human requires the amount of training data we are already giving the models." Well, humans are also trained differently. We interact with other humans in real time and get immediate feedback on their responses. We don't just learn by reading through reams of static information. We talk to people. We get into arguments. And so on. Maybe the ideal way to train an AI is to have it interact with lots of humans, so…

It's not just humans we interact with - it's reality. For an LLM all "facts" are socially-determined, for a human they are not

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#37

Post by smokel »

vouaobrasil wrote: post #17
omneity wrote: post #14

Earlier quoted context omitted.

What makes you think this technology is out of control? If anything a 1T param model would only run in like 5-10 computers in this world. As much control as it gets. By the way this is doomer rhetoric. Bashing scientific advancements as dangerous or useless by trying to attach vacuously scary ideas that are as speculative as they are outlandish.

It's hardly vacuous. What about these scientific advancements? 1. Fossil fuel extraction -- we are killing our planet. 2. Plastic chemistry -- there are 5 trillion pieces of plastic in the ocean killing seabirds, turtles, fish ( https://doi.org/10.1371/journal.pone.0111913 ) 3. Concrete engineering -- building cities that move people away from a connection with nature 4. Veterinary medicine -- supporting global meat…

As much as I'd like to agree with your sentiment, it is a rather flawed argument to state that if some elements of a set have a property, that all elements must have that property.

There are scientific breakthroughs such as the proof of Fermat's theorem [1], for which I find it very hard to envision a way in which it will kill our planet.

[1] https://en.m.wikipedia.org/wiki/Wiles%27s_proof_of_Fermat%27...

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#38

Post by mcosta »

vouaobrasil wrote: post #17
omneity wrote: post #14

Earlier quoted context omitted.

What makes you think this technology is out of control? If anything a 1T param model would only run in like 5-10 computers in this world. As much control as it gets. By the way this is doomer rhetoric. Bashing scientific advancements as dangerous or useless by trying to attach vacuously scary ideas that are as speculative as they are outlandish.

It's hardly vacuous. What about these scientific advancements? 1. Fossil fuel extraction -- we are killing our planet. 2. Plastic chemistry -- there are 5 trillion pieces of plastic in the ocean killing seabirds, turtles, fish ( https://doi.org/10.1371/journal.pone.0111913 ) 3. Concrete engineering -- building cities that move people away from a connection with nature 4. Veterinary medicine -- supporting global meat…

Well, I don't care about nonhuman animals.

And being sincere, I don't care about 99% of human animals.

I cry for the deaths of my loved ones, and forget the rest. It is the only way to remain sane.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#39

Post by Andrex »

Pompidou wrote: post #28
jojobas wrote: post #24

Earlier quoted context omitted.

The first humans were kinda dumb. Yet by interacting with the world and each other, humans got smart. I wonder if neural network instances can interact with a simulated environment and each other, they could become smart enough for practical purposes. Kinda like alpha go zero but with "life".

> The first humans were kinda dumb How can you assert this ? Do you have any evidence?

Even tracing it back to the origin of Homo erectus, the line would still be too blurry. I don't think "the first humans" can actually mean anything in a science-based discussion.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#40

Post by Andrex »

Zanni wrote: post #31
jojobas wrote: post #30

Earlier quoted context omitted.

Can't tell if you're joking or serious. In the latter case, the first humans (H. Habilis) had about 1/2 of H. Sapiens brain to work with, and a much smaller fraction of neocortex. If that doesn't satisfy you, let's say I was speaking about some sort of human ancestor before that, which would have been about as dumb as chimps, unless you require proof of their dumbness as well.

Not your original objector, but ... I was willing to accept that early humans are dumb, but your explanation relies on evolution and not social interaction.

Exactly. Humans didn't learn their way out of only having half a brain, more the opposite.
Post reply on HN