401–410 of 719 posts
It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…
I'm not surprised-- I recently suggested that someone might try to pull an Aaron Swartz with the LLAMA weights (i.e., release them in an uncontrolled way similar to how Aaron attempted to release the JSTOR database). It's quite misleading for FB to claim that they are being so open, but then hoard the weights and only release it to a few academics. If the paper is to be believed, this is a major development, allowing…
if aaron had attempted to release the jstor database he would have succeeded
Earlier quoted context omitted.
On the flip side of subhuman treatment of humans, we have useful legal fictions like corporate personhood. It's going to be pretty rough for a while, particularly for nontechnical judges, to sort all of this out. We're almost definitely going to see multiple rulings far more bizarre than Citizens United ruling that limiting corporate donations limits the free-speech rights of the corporation as a person. I'm not a la…
Good points but it’s worth clarifying that this is not what the Citizens United decision said. It clarified that the state couldn’t decide that the political speech of some corporations (Hillary the Movie produced by Citizens United) was illegal speech and speech from another corporation (Farenheit 9/11 by Dog Eat Dog films and Miramax) was allowed. Understood this way it seems obvious on free speech grounds, and in…
Earlier quoted context omitted.
I don't think that's right - even if you had the full source code for either of those, it's extremely unlikely you'd be able to build them on your own machine.
there's 99% complete leaked windows xp source code that people have managed to compile
It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to. [1]: https://boards.4channel.org/g/thread/9184826…
Why do 4chan users go out of their way to be so offensive in their posts?
Earlier quoted context omitted.
Some years ago someone that worked at Microsoft told me he didn't think any individual engineer who already works on Windows could ever get Windows building by themselves with just the code.
That just shows their bias. If the code is complete, it's only a matter of time to figure out how to build it. May not take hours, but determined engineer should be able to figure it out.
Earlier quoted context omitted.
My understanding is that weights are normally stored as pickled python blobs, which means arbitrary code execution as they are unpickled.
A few months ago I made a small library to sanitize pytorch checkpoints, here it is: https://github.com/kir-gadjello/safer_unpickle The usage boils down to import safer_unpickle from safer_unpickle safer_unpickle.patch_torch_load() This overrides default torch unpickler with a relatively safe one. Hope this helps.
Earlier quoted context omitted.
There is a neat potential speedup here for the case where the bandwidth to your model weights is the limiting factor. If you have a guess what the model will output, then you can verify that your guess is correct very cheaply, since you can do it in parallel. That means there is the possibility to have a highly quantized small model in RAM, and then use the big model only from time to time. You might be able to get a…
This is an interesting concept, could you share a paper or some writeup about this?
The potential for speedup according to their paper is closer to 2x than 10x however.
Earlier quoted context omitted.
Electricity costs are basically irrelevant because the cards are so expensive. A100 cards consume 250w each, with datacenter overheads we will call it 1000 kilowatts for all 2048 cards. 23 days is 552 hours, or 552,000 kilowatt hours total. Most dataceneters are between 7 and 10 cents per kilowatt hour for electricity. Some are below 4. At 10 cents, that's $53,000 in electricity costs, which is nothing next to $30 mi…
> Electricity costs are basically irrelevant because the cards are so expensive. You mean in terms of money. I think this is exactly the problem that we have in CS, nobody really cares about CO2.
Earlier quoted context omitted.
Used it to download linux distro images back when the size of an install CD was huge. Good times.
Now that I think about it I wonder why we don't see it being used to distribute packages for linux distros. Seems more flexible than the current mirror system.