41–50 of 67 posts
So perhaps maybe the goal should be to make better AI’s rather than “more powerful” toxic people-on-the-internet simulators?
Earlier quoted context omitted.
If you're looking to make a tweet generator, sure. If you're looking to harness the power of facts, no.
The main issue might be that the training set may get polluted by a lot of AI generated text (content that is not supposed to be there, but is there because AI-generated content is becoming widespread online).
I have a probably really stupid question but I'm going to ask it because I really don't know the answer. What's stopping someone from starting a distributed effort to train an open source ChatGPT model, like Folding@home but for AI? Is it a technical limitation with how models are trained, difficulty of coordination, something else?
Training in parallel on more then one GPU is pretty hard. Distributing the training over many random computers and adding at least some kind of validation, seems to kill the performance. So it would always be easier to rend cloud GPUs.
I have a probably really stupid question but I'm going to ask it because I really don't know the answer. What's stopping someone from starting a distributed effort to train an open source ChatGPT model, like Folding@home but for AI? Is it a technical limitation with how models are trained, difficulty of coordination, something else?
There is not enough written text to scale a model like ChatGPT at even 10x its current parameters, so improvements will have to come from optimizations and/or longer training (at diminishing returns).
Is there a point where those AI models could plateau and adding more parameters or data will not improve them anymore?
There's also a consequence of performance by adding more params. The inference time will be longer and even just training the model will take longer and won't be able to run as many epochs in an efficient manner.
I have a probably really stupid question but I'm going to ask it because I really don't know the answer. What's stopping someone from starting a distributed effort to train an open source ChatGPT model, like Folding@home but for AI? Is it a technical limitation with how models are trained, difficulty of coordination, something else?
The models are extremely large to ship around, sync, and load on consumer hardware over the internet. Maybe there are better ways to build and train a model for a distributed use case though.
Translation: nVidia is talking up the hype that can increase its stock price (now that the cryptocurrency hype is over)
Hah
(great time to use https://lobste.rs/)
Earlier quoted context omitted.
If you're looking to make a tweet generator, sure. If you're looking to harness the power of facts, no.
So interesting that people think facts are the only thing that's valuable. Your comment isn't a fact, does that mean it's worthless?