61–70 of 143 posts
How does LLaMA handle fast fine-tuning? Are they using transformer adapters for it?
[1] https://github.com/huggingface/transformers/pull/21955
[2] https://github.com/oobabooga/text-generation-webui/commit/90...
If anyone is interested in running this at home, please follow the llama-int8 project [1]. LLM.int8() is a recent development allowing LLMs to run in half the memory without loss of performance [2]. Note that at the end of [2]'s abstract, the authors state "This result makes such models much more accessible, for example making it possible to use OPT-175B/BLOOM on a single server with consumer GPUs. We open-source our…
Hmmm, the Github repo suggests that you might be able to run the 65B model on a single A100 80gb card. At the moment, the spot price on Google cloud for this card is $1.25/hour which makes it not so crazy expensive...
If anyone is interested in running this at home, please follow the llama-int8 project [1]. LLM.int8() is a recent development allowing LLMs to run in half the memory without loss of performance [2]. Note that at the end of [2]'s abstract, the authors state "This result makes such models much more accessible, for example making it possible to use OPT-175B/BLOOM on a single server with consumer GPUs. We open-source our…
I feel like we're less than a decade away from being able to hook LLMs into gaming. How incredible would it be to have NPCs driven by LLM?
Earlier quoted context omitted.
Would you mind publishing your notes/learnings once you gain enough understanding of this model?
Absolutely! I'll make sure to leave a comment here for you whenever something gets written up so you don't miss it. Getting "as good as davinci" on a single A100 is groundbreaking work. Facebook and the community should both be credited here -- maybe llama-int8 would've been created even if the model hadn't leaked, but I don't think it would've happened so quickly. Everyone is doing phenomenal work, and it's so amazi…
I see vast.ai listing interruptible instance with a single A100 80GB at $1/hour which is pretty reasonable. ChatGPT plus is $20/month which would be roughly 20 hours of use and I wont't be lectured like I'm in a kindergarten or something.
A bonus point would be to make the writeup accessible for AI challenged developers. Asking for a friend.
Is this the full model or just the weights? [EDIT]: are there checksums available? [EDIT2]: MD5 signatures seem to be included for all models in checklist.chk files next to them And there's also what the author mentions: the magnet file he provides in his README does seed immediately on the download when loaded in a bt app which is usually a good sign that the files are correct.
I womder, could Facebook take legal action here? While some (most of) the data used to train the model is copyrighted, I don't think the model is. It's the result of a mathematical process applied to a series of facts and works with no more creativity put onto them.
Not too familiar with the drama but I believe what happened was that someone with access leaked the torrent used to download the weights. In a legal sense this would be similar to someone say leaking a Google Drive link containing prop information that was only intended to be shared with vendors.
If anyone is interested in running this at home, please follow the llama-int8 project [1]. LLM.int8() is a recent development allowing LLMs to run in half the memory without loss of performance [2]. Note that at the end of [2]'s abstract, the authors state "This result makes such models much more accessible, for example making it possible to use OPT-175B/BLOOM on a single server with consumer GPUs. We open-source our…
I feel like we're less than a decade away from being able to hook LLMs into gaming. How incredible would it be to have NPCs driven by LLM?
https://news.ycombinator.com/item?id=34478503
I have long wished for less linear stories in video games, where branching narrative (a la Choose Your Own Adventure) is one possible way to give the player agency. The problem is, true branches are expensive, because you end up writing a bunch of content the player never experiences.
I see a lot of potential, but it's going to take a different kind of craftsmanship, and likely many iterations, to realize something more than a novelty.
Earlier quoted context omitted.
That definition would apply to almost anything software produces ^^; We can already have different licenses for compiled binaries vs the source. Also the output of ML seems to belong to whoever pressed the generate button atm.
> That definition would apply to almost anything software produces Not really. The reason software can be copyrighted at all is because the actual code (and resulting object code) is creative. Courts have named this threshold the "Structure, sequence and organization" of the work. ML models don't follow any creative SSO the way actual code does. > Also the output of ML seems to belong to whoever pressed the generate…
Earlier quoted context omitted.
You only get a single month-long window to spend the credit! And I'm sure not going to spend any of my own money on prompting experiments. I might be suffering from FOMO to some degree, I've just got to tell myself that this won't have been the only time model weights get leaked!
> And I'm sure not going to spend any of my own money on prompting experiments. This certainly sounds a lot like whining that others aren’t doing the work you yourself don’t want to do.
I'm not in a position to put in any meaningful work towards optimising this model for lower-end hardware, or working on the tooling/documentation/user experience.