51–60 of 143 posts
Nice. You need a 28GB GPU, so it's not exactly something people can run on their laptop. Everybody's server costs are about to go the roof.
Earlier quoted context omitted.
Essentially a computer neural network is just a lot of addition (and matrix multiplication) of floating point numbers. The parameters are the "strength" or "weights" of the connections between neurons on different layers and the "bias" of each neuron. If neuron Alice is connected to neuron Bob and Alice has a value of 0.7, and the weight of Alice's connection to bob is 0.5, then the value sent from Alice to Bob is 0.…
To add to this excellent reply, I'll also point out that the reason folks want the weights is that they are the result of a massive search operation, akin to finding the right temperature to bake a cake from all possible floats. It takes a lot of wall clock time, and a lot of GPU energy, and a lot of input examples and counter-examples to find the "right" numbers. Thus, it really is better -- all things being equal -…
...for each of 13 billion (for a model with that many parameters) different cakes, except that they aren’t like cakes because the “best" temperature for each depends on the actual temperatures chosen for the others.
Earlier quoted context omitted.
(I work on llama-dl.) We’re fighting back against the DMCA requests on the basis that NN weights aren’t copyrightable. This thread has details: https://news.ycombinator.com/item?id=35393782 I don't think you have to worry about Facebook going after you. The worst that will happen is that they issue a DMCA, in which case your project gets knocked offline. I don’t think they’ll be going the RIAA route of suing individu…
Lemme save whoever is donating the legal here the time: model weights are definitely copyrightable.
Nice. You need a 28GB GPU, so it's not exactly something people can run on their laptop. Everybody's server costs are about to go the roof.
From the git repo: > This conversion command needs around 60 GB of CPU RAM. Ok. I don't have that. Has/will someone release the full weights with the deltas applied?
Could someone explain how to test this? Applying the delta conversion requires 60GB of CPU RAM. Do you just have 60GB RAM on your machine?
It's actually very impressive. I gave it the task of converting a query and an OpenAPI spec into an API call, and it worked! I've not been succesful in getting GPT-3.5 to do this without rambling on about the reasoning for its decision.
Usually if I want code from the GPT family I always add "Just show me the code, no extra words or explanation" in the end of the prompt, and it works 99% of the time. Edit: just finished the conversion of Vicuna myself now and been doing some light testing, seems to work in ~80% of the cases for it, not as high success-rate as with GPT for sure. Probably there is a better way of structuring the prompt for Vicuna.
Earlier quoted context omitted.
Usually if I want code from the GPT family I always add "Just show me the code, no extra words or explanation" in the end of the prompt, and it works 99% of the time. Edit: just finished the conversion of Vicuna myself now and been doing some light testing, seems to work in ~80% of the cases for it, not as high success-rate as with GPT for sure. Probably there is a better way of structuring the prompt for Vicuna.
You can just say “no explain” and it won’t explain. Don’t waste tokens patronizing gpt lol
def fibonacci(n): if n <= 1: return n else: return fibonacci(n-1) + fibonacci(n-2)
Earlier quoted context omitted.
Not a lawyer, but that still feels like dubious territory. I would still be on the hook for acquiring the original download, which Facebook has been launching dmca takedown requests for the llama-dl project.
It's fairly similar to a ROM patch in the video game space, which has mostly stood the test of time.
Nice. You need a 28GB GPU, so it's not exactly something people can run on their laptop. Everybody's server costs are about to go the roof.