51–60 of 64 posts
I continue to find Mark impressive. I think there's something quite true about the "trying to create God" remark.
Only the guy not leading says stuff like this. You can be sure he’d be up there convincing Congress that moats need to be built for his competitors and building opaque systems if fortune favored differently.
Earlier quoted context omitted.
just putting weights on hug is not enough for a model to be open. it is like sharing the binaries of a software. they should put their entire code on github and show how it was trained, architecture and everything else. also, the most important thing about a model is not it's architecture, but the dataset it was trained on. that is the "it" in a model.
I am not sure why people keep pretending like this is possible or keep asking it to happen. You know that distributing all the data would be illegal copyright infringement, right? Also, its not like they could distribute the (petabytes?) worth of data to every rando who asks. Do you just not want open weight models to exist, and you are using a fake call for it to be more open, so as to get this stuff all shut down,…
We want words to mean things. model available != open source, If you go to a restaurant and order the beef and you get the fish, what's wrong, it's still food, just eat it. Don't be pedantic, otherwise the cook isn't going to make food anymore. If you're 10 and your parents made you dinner that might pass, but if you're an adult and go to a restaurant where you're paying with your own hard earned money, that's not how it works.
They don't have to distribute the data directly, Stability AI has said they used the LAION-5B Dataset. Just like I could say I read Harry Potter without giving you a copy, and can say that it says "the boy who lived" without violating copyright.
Because Zuck's AI is so open
Wait, but... it is. Am I missing something, or inferring sarcasm where there is none?
Earlier quoted context omitted.
Open source != open weights != open training set. Of course, GP is ignoring that Meta is far more open than its direct competitors (MSFT, GOOG, OpenAI).
I agree that open source or just “open” is the wrong term here although it’s hard to apply it even to a project where the training data and everything else are shared. “Free” or “libre” might be closer since the end product is free to use and modify. It’s still fuzzy though, are weights code or are they an API? Oddly enough they seem to be the embodiment of the interchangeability of code and data perhaps due to neura…
I continue to find Mark impressive. I think there's something quite true about the "trying to create God" remark.
Earlier quoted context omitted.
I am not sure why people keep pretending like this is possible or keep asking it to happen. You know that distributing all the data would be illegal copyright infringement, right? Also, its not like they could distribute the (petabytes?) worth of data to every rando who asks. Do you just not want open weight models to exist, and you are using a fake call for it to be more open, so as to get this stuff all shut down,…
Anyone who understands the difference between model available and open source would realize they're different things, and then also be smart enough to realize that randos arguing about the definition isn't what's going to sway someone as to wether they'd release the model or the source used to create them. You think Mark Zuckerberg, or anyone else with the $100s of million dollars to spend on training is reading thes…
Its not because he said things.
Instead it is that (if implemented) the things that he is advocating for would actually significantly reduce the amount of open-whatever-you-want-to-call-it models available as most of it would all get shut down.
If these people don't want these models to exist and they are just using this concern trolling to get rid of all these AI models, then I wish they would just say so openly.
Because that would be the result of their positions. It would be a destruction of most of the more-open AI models that we have available today and an end to the open-whatever AI industry.
If thats what you want, just say it, and then we can talk about the pros and cons of a completely centralized AI ecosystem that is only controlled by OpenAI, Google, and the government and how if the destruction of the AI startup space that is reliant on the models given out by meta/ect is good or bad.
Earlier quoted context omitted.
I agree that open source or just “open” is the wrong term here although it’s hard to apply it even to a project where the training data and everything else are shared. “Free” or “libre” might be closer since the end product is free to use and modify. It’s still fuzzy though, are weights code or are they an API? Oddly enough they seem to be the embodiment of the interchangeability of code and data perhaps due to neura…
it's not fuzzy. weights are binary blobs. we know what binaries are, source code gets compiled into binaries. photoshop is a binary. firmware are binary blobs. that llama3.ggml outputs text when using a wrapper program vs photoshop launches a program directly doesn't make anything fuzzy.
Earlier quoted context omitted.
it's not fuzzy. weights are binary blobs. we know what binaries are, source code gets compiled into binaries. photoshop is a binary. firmware are binary blobs. that llama3.ggml outputs text when using a wrapper program vs photoshop launches a program directly doesn't make anything fuzzy.
Not really, those weights are the same ones that are there during training getting modified by backprop. You can take them and continue training the model in order to produce different behaviour. You can also modify them directly like with LoRA or steering. You can also make bigger modifications like expanding the layers or removing nodes. It’s not a frozen binary, those numbers aren’t obfuscated or optimized into so…
Earlier quoted context omitted.
Anyone who understands the difference between model available and open source would realize they're different things, and then also be smart enough to realize that randos arguing about the definition isn't what's going to sway someone as to wether they'd release the model or the source used to create them. You think Mark Zuckerberg, or anyone else with the $100s of million dollars to spend on training is reading thes…
> I was going to release Llama4, but because PunchTornado said pedantic things, y'know what? I'm not gonna. Its not because he said things. Instead it is that (if implemented) the things that he is advocating for would actually significantly reduce the amount of open-whatever-you-want-to-call-it models available as most of it would all get shut down. If these people don't want these models to exist and they are just…
Trying to paint me into a corner with a false dichotomy, that if I want it to be called model available instead of open source, and that in doing so, I want a dystopian future where megacorps have AGI and we're all at their mercy isn't going to work. I don't want the models to not be available, what I want is to see what's being used to create the models, the source, if you will, and thus the alignment side of the source, and why it won't answer "how do I make cocaine", as if I'm going to go off and somehow become Tony Montoya. We may never get that, but that's what calling it open source and have it mean that would entail. Making the models available to the plebs is a generous thing for Mark to do, but that doesn't mean we can't ask for more. Or maybe we shouldn't; and just be grateful for his benevolence, because it's not like OpenAI has done the same with ChatGPT. I will give them credit for Whisper though.