Training for one trillion parameter model backed by Intel and US govt has begun
1–10 of 267 posts
Re: Training for one trillion parameter model backed by Intel and US govt has begun
#2Had to happen eventually.
Re: Training for one trillion parameter model backed by Intel and US govt has begun
#3Not surprising. Despite the enormous energy costs and the threat to humanity by creating technology that we can't control, governments and corporations will build bigger and more sophisticated models just because they have to do so to compete.
It is the prisoner's dilemma: we end up with a super-advanced AI that will disrupt and make society worse because entities are competing where the metric of success is short-term monetary gain.
It's ridiculous. Humanity should give up this useless development of AI.
Re: Training for one trillion parameter model backed by Intel and US govt has begun
#4Not bad for a start by the US government. A bit more than half as powerful as the 1.7 trillion parameter GPT4 model.
Re: Training for one trillion parameter model backed by Intel and US govt has begun
#5Isn't GPT-4 already over 1T parameters? And GPT-5 should be even "an order of magnitude" bigger than GPT-4...
Re: Training for one trillion parameter model backed by Intel and US govt has begun
#6The solution won't be just "bigger". A model with a trillion parameters will be more expensive to train and to run, but is unlikely to be better. Think of the early days of flight, you had biplanes; then you had triplanes. You could have followed that farther, and added more wings - but it wouldn't have improved things.
Improving AI will involve architectural changes. No human requires the amount of training data we are already giving the models. Improvements will make more efficient use of that data, and (no idea how - innovation required) allow them to generalize and reason from that data.
Re: Training for one trillion parameter model backed by Intel and US govt has begun
#7Not bad for a start by the US government. A bit more than half as powerful as the 1.7 trillion parameter GPT4 model.
A bit more than half the size, it remains to be seen how powerful it is. There's clearly a non-linear relationship to model size, and it's also clear that it's hard to assess the power of these models anyway.
Re: Training for one trillion parameter model backed by Intel and US govt has begun
#8Not surprising. Despite the enormous energy costs and the threat to humanity by creating technology that we can't control, governments and corporations will build bigger and more sophisticated models just because they have to do so to compete. It is the prisoner's dilemma: we end up with a super-advanced AI that will disrupt and make society worse because entities are competing where the metric of success is short-te…
It might be dangerous, but how is it useless? There is also an enormous positive gain to be had in terms of autonomous discovery, science progress and so on. And prisoner's dilemma means "we" will not stop pursuing this, even of there is a net negative benefit to humanity overall.
Re: Training for one trillion parameter model backed by Intel and US govt has begun
#9Not bad for a start by the US government. A bit more than half as powerful as the 1.7 trillion parameter GPT4 model.
A bit more than half the size, it remains to be seen how powerful it is. There's clearly a non-linear relationship to model size, and it's also clear that it's hard to assess the power of these models anyway.
Ah, an important distinction to make. Good point.
Re: Training for one trillion parameter model backed by Intel and US govt has begun
#10Isn't GPT-4 already over 1T parameters? And GPT-5 should be even "an order of magnitude" bigger than GPT-4...
Yes, the article even mentions that. Guess they went with that headline to attract readers.