OpenAI needs to release a new model setting a new capabilities highpoint. This is existential for them now.
Open source AI is the path forward
131–140 of 936 posts
Re: Open source AI is the path forward
#132Earlier quoted context omitted.
I also think that something like Chromium is a better analogy for corporate open source models than a grassroots project like Linux is. Chromium is technically open source, but Google has absolute control over the direction of it's development and realistically it's far too complex to maintain a fork without Googles resources, just like Meta has complete control over what goes into their open models, and even if they…
I think you’re right from the perspective of an individual developer. You and I are not about to fork Chromium any time soon. If you presume that forking is impractical then sure, the right to fork isn’t worth much. But just because a single developer couldn’t do it doesn’t mean it couldn’t be done. It means nobody has organized a large enough effort yet. For something like a browser, which is critical for security,…
If Microsoft can't do it, then we can reasonably conclude that it can't be done for any practical purpose. Discussing infinitesimal possibilities is better left to philosophers.
Re: Open source AI is the path forward
#133Related:
Llama 3.1 Official Launch
Re: Open source AI is the path forward
#134The value of AI is in the information used to train the models, not the hardware.
Re: Open source AI is the path forward
#135> Today we’re taking the next steps towards open source AI becoming the industry standard. We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, Why do people keep mislabeling this as Open Source? The whole point of calling something Open Source is that the "magic sauce" of how to build something is publicly available, so I could built it myself if I have the means. But without the training d…
The codebase to do the training is way less valuable than the weights for the vast majority of people. Releasing the training code would be nice, but it doesn't really help anyone but Meta's direct competitors.
If you want to train on top of Llama there's absolutely nothing stopping you. Plenty of open source tools to do parameter optimization.
Re: Open source AI is the path forward
#136The truth is we need both closed and open source, they both have their discovery path and advantages and disadvantages, there shouldn’t be a system where one is eliminated over the other. They also seem to be driving each other forward via competition.
Re: Open source AI is the path forward
#137I am not deep into llms so I ask this. From my understanding, their last model was open source but it was in a way that you can use them but the inner working were "hidden"/not transparent. With the new model, I am seeing alot of how open source they are and can be build upon. Is it now completely open source or similar to their last models ?
It's intrinsic to transformers that the inner workings are largely inscrutable. This is no different, but it does not mean they cannot be built upon.
Gradient descent works on these models just like the prior ones.
Re: Open source AI is the path forward
#138"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…
Open training dataset + open steps sufficient to train exactly the same model.
They can't release training dataset if it was illegally scrapped all over the web without permission :) (taps head)
Re: Open source AI is the path forward
#139Thanks to Meta for their work on safety, particularly Llama Guard. Llama Guard 3 adds defamation, elections, and code interpreter abuse as detection categories.
Having run many red teams recently as I build out promptfoo's red teaming featureset [0], I've noticed the Llama models punch above their weight in terms of accuracy when it comes to safety. People hate excessive guardrails and Llama seems to thread the needle.
Very bullish on open source.
Re: Open source AI is the path forward
#140> Our adversaries are great at espionage, stealing models that fit on a thumb drive is relatively easy, and most tech companies are far from operating in a way that would make this more difficult. Mostly unrelated to the correctness of the article, but this feels like a bad argument. AFAIK, Anthropic/OpenAI/Google are not having issues with their weights being leaked (are they?). Why is it that Meta's model weights a…
We have no way of knowing whether nation-state level actors have access to those weights.