161–170 of 334 posts
Earlier quoted context omitted.
Great news! Cloud instances energy usage is included in their price, and because they're remote and transient it's impossible to permanently damage them.
I think the equivalent of being not careful and getting a dent in this context is to leave it open to the internet and having a bitcoin miner installed.
Earlier quoted context omitted.
The repo seems to imply that it matches GPT-2, so I imagine any analyses of GPT-2 will give you a good idea.
I’m not easily finding GPT-2 use cases. Any query guidance?
Earlier quoted context omitted.
afaik, gpt-4 is mostly rumours so far, same thing for the 1.5T number. gpt-4 is suerly coming.
Maybe it will be called GPT-XP by then, with Microsoft owning half of it.
And then three years later GPT-11 will be required to run the latest games.
Earlier quoted context omitted.
>> GPT-3 took 355 years to train > Could this be distributed? Put all those mining GPUs to work. Nope. It's a strictly O(n) process. If it weren't for the foresight of George Patrick Turnbull in 1668, we would not be anywhere close to these amazing results today.
I couldn't find any references to George Patrick Turnbull. If that an ancestor of yours? If so, the comment seems rather subjective.
Should I use this or minGPT?
It says it needs 8XA100 40GB node. What is that and where do I acquire it?
Could someone else train this and then send me the model? What would be required to run it as opposed to training it?
Earlier quoted context omitted.
Could this be distributed? Put all those mining GPUs to work. A lot of people like participating in public projects like this. I would!
>> GPT-3 took 355 years to train > Could this be distributed? Put all those mining GPUs to work. Nope. It's a strictly O(n) process. If it weren't for the foresight of George Patrick Turnbull in 1668, we would not be anywhere close to these amazing results today.
Excuse my ignorance but what can a layman do with this?
Earlier quoted context omitted.
After training don't you have to keep it running if you want to use it?
Just download the model and run it on something much smaller and cheaper. Bigger models like GPT-J are a bit of a pain to run, but GPT2-sized models run just fine on consumer GPUs.
For an AI noob like me: can you use spot instances to train models? They are about 1/3rd the price on AWS compared to on demand ones, so it'd make a significant difference.
If you are going to run a long training job, ensure you are creating checkpoints. Be sure to use persistent storage, EBS and ensure that you check the option that it doesn't get deleted if the instance is stopped, so your checkpoint remain in the disk and you can easily restart.
I haven't tried it but prices here are much cheaper. https://vast.ai/#pricing
Earlier quoted context omitted.
GPT and human brain ( at least the language / speech part ) have nothing in common. We, as humans, do not use language in a generative way, is derived from a higher or very low level of abstraction ( intentions, emotions, etc ) and is explictly use for communicating something. Even this text is based on previous knowledge, saved in an abstract way, and while writing this I must follow the synthax of the language or w…
These are conceptual "differences" that don't actually explain the mechanics of what's going on. For all you know "motivation", "intentions", etc. are also just GPT-like subsystems, in which case the underlying mechanics are not as different as you imply.
Whatever humans have it is many orders of magnitude better…