NanoGPT
181–190 of 334 posts
Re: NanoGPT
#182Earlier quoted context omitted.
These are conceptual "differences" that don't actually explain the mechanics of what's going on. For all you know "motivation", "intentions", etc. are also just GPT-like subsystems, in which case the underlying mechanics are not as different as you imply.
If it were gpt-like sub systems, humans would be emitting MWs of power instead of the 100W now. Whatever humans have it is many orders of magnitude better…
That's the hardware it runs on, not the software architecture of GPT. I could equally say that transistors are faster than synapses by the same ratio that marathon runners are faster than continental drift.
Re: NanoGPT
#183Thank you Andrej Karpathy for the work on ai and gpt models. It really helped me solve a problem as entrepreneur. I started making first few grand from ai.
Re: NanoGPT
#184Earlier quoted context omitted.
Just download the model and run it on something much smaller and cheaper. Bigger models like GPT-J are a bit of a pain to run, but GPT2-sized models run just fine on consumer GPUs.
What’s required to run the model?
The biggest GPT2 (1.5B params) takes about 10GB VRAM, meaning it runs on a RTX 2080 TI, or the 12GB version of the RTX 3080
Re: NanoGPT
#185This is really good, and I was really excited by it but then I read: > running on a single 8XA100 40GB node in 38 hours of training This is a $40-80k machine. Not a diss, but I would love to see an advance that would allow anyone with a high end computer to be able to improve on this model. Before that happens this whole field is going to be owned by big corporations.
It's a $33/hour machine on AWS, so about $1250 for one training run. Not cheap, but easily in the reach of startups and educational or research institutions. Edit: or about $340 if you get the 8xA100 instance from lambdalabs, in the realm of normal hobby spending
If you're doing something new/ custom (which you presumably are if you aren't using someone else's prebuilt model), it could take a lot of runs to figure out the best training data and finetune settings.
(I assume. I've never worked with GPT, but have done similar work in other domains).
Re: NanoGPT
#186Earlier quoted context omitted.
afaik, gpt-4 is mostly rumours so far, same thing for the 1.5T number. gpt-4 is suerly coming.
Maybe it will be called GPT-XP by then, with Microsoft owning half of it.
In my imagination, OpenAI does what Bungie did when MS bought them, and open-sources what used to be their crown jewels.
That said, GPT-AlephOne only makes sense if there's a preceding GPT-∞.
Re: NanoGPT
#187Re: NanoGPT
#188I could not find any sample (prompt and results). Can anyone provide samples of it's quality, even if it is in a narrow field of knowledge or specific use case?
I tried GPT2, GPT-J 6B and GPT-NeoX 20B (implementation by Fabrice Bellard at textsynth.com/playground.html) but I could not find any production-quality scenario yet, only cherry-picked simple cases.
Re: NanoGPT
#189Earlier quoted context omitted.
Does anyone know the main differences between GPT-2 and GPT-3? Are there significant architectural changes, or is the advancement primarily from training?
If you google "GPT-2 vs GPT-3" you'll find lots of overviews and comparisons, like: * https://www.kdnuggets.com/2021/02/gpt2-gpt3-openai-showdown.... * https://bakztfuture.substack.com/p/the-chasm-between-gpt-2-a...
Thanks. Sounds like they 10x'ed the number of parameters, which made some "magic leap" that isn't yet well understood, and fed it more data to train it on more specialized domains.
Re: NanoGPT
#190Earlier quoted context omitted.
Yeah that's the issue I was thinking of, how to get it to summarise large documents. Has anyone any ideas?
People have had some success with the following process: Divide your 30,000 word document into a hundred 300 word chuncks. For each chunk, give as input: Please summarize the following text into 50 words: [chunk] Join all the outputs together, and you now have a shorter document. Repeat the process recursively. You can improve the results by doing the process again, but this time giving some context: Please summarize…
You can also use "Please suggest a section title for the following text".
Then that title can be used in the 2nd round, for example using a query of the form "The following is an extract from the Introduction section of a document about The benefits and disadvantages of nuclear power in sweden:"