Open source AI is the path forward • Hacker News Board

Re: Open source AI is the path forward

#571

Post by Der_Einzige » Tue, Jul 23, 2024, 11:26 PM UTC

meowtimemania wrote: post #526

war321 wrote: post #450

Even if it's just open weights and not "true" open source, I'll still give Meta the appreciation of being one of the few big AI companies actually committed to open models. In an ecosystem where groups like Anthropic and OpenAI keep hemming and hawing about safety and the necessity of closed AI systems "for our sake", they stand out among the rest.

Why would openai/anthropic's approach be more safe? Are people able to remove all the guard rails on the llama models?

Yes, it's trivial to remove guardrails from any open access model:

https://www.lesswrong.com/posts/jGuXSZgv6qfdhMCuJ/refusal-in...

https://huggingface.co/failspy/Llama-3-70B-Instruct-ablitera...

Re: Open source AI is the path forward

#572

Post by mvkel » Tue, Jul 23, 2024, 11:30 PM UTC

JamesBarney wrote: post #259

lolinder wrote: post #94

Earlier quoted context omitted.

Source versus weights seems like a really pedantic distinction to make. As you say, the training code and training data would be worthless to anyone who doesn't have compute on the level that Meta does. Arguably, the weights are source code interpreted by an inference engine, and realistically it's the weights that someone is going to want to modify through fine-tuning, not the original training code and data. The fa…

https://llama.meta.com/llama3_1/use-policy/ The acceptable use policy is seems fine. Don't use it to break the law, solicit sex, kill people, or lie.

This is like saying "You have the right to privacy. The police can tap your phone, but you have nothing to worry about as long as you're not breaking the law."

"we're open source, you can use it for anything you can imagine. But you can't use it for these specific things."

Then there's the added rub of the source not really being source code, but a CSV file.

That's fine. If you want to set that expectation, great! But don't call it open source.

Re: Open source AI is the path forward

#573

Post by stavros » Tue, Jul 23, 2024, 11:32 PM UTC

proteal wrote: post #554

larodi wrote: post #515

Earlier quoted context omitted.

Indeed, since when the deliverable being a jpeg/exe, which is similar to what the model file is, is considered the source? it is more like open result or freely available vm image, which works, but has its core FS scrambled or crypted. Zuck knows this very well and it does him no honour to speak like, and from his position this equals attempt ate trying to change the present semantics of open source. Of course, other…

Even if everything was released how you described, what good would that really do for an individual without access to heaps of compute? Functionally there seems to be no difference between open weights and open compute because nobody could train a facsimile model. Furthermore, all frontier models are inscrutable due to their construction. It’s wild to me seeing people complain semantics when meta dropped their model…

That's not the discussion. We're talking about what open source is, and it's having the weights and the method to recreate the model.

If someone gives me an executable that I can run for free, and then says "eh why do you want the source, it would take you a long time to compile", that doesn't make it open source, it just makes it gratis.

Re: Open source AI is the path forward

#574

Post by yangcheng » Tue, Jul 23, 2024, 11:32 PM UTC

causal wrote: post #41

the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

"Open weights" is a more appropriate term but I'll point out that these weights are also largely inscrutable to the people with the code that trained it. And for licensing reasons, the datasets may not be possible to share. There is still a lot of modifying you can do with a set of weights, and they make great foundations for new stuff, but yeah we may never see a competitive model that's 100% buildable at home. Edit…

latest llama 3.1 is in a different repo, https://github.com/meta-llama/llama-models/blob/main/models/... , but yes, the code is shared. It astonishing that in software 2.0 era, powerful applications like llama has only hundreds of lines of code, and most work hidden in training data. Source code alone is no longer that informative as Software 1.0

Re: Open source AI is the path forward

#575

Post by cageface » Tue, Jul 23, 2024, 11:42 PM UTC

pwdisswordfishd wrote: post #311

cs702 wrote: post #74

> We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models. Bravo! While I don't agree with Zuck's views and actions on many fronts, on this occasion I think he and the AI folks at Meta deserve our praise and gratitude. With this release, they have brought the cost of pretraining a frontier 400B+ parameter model to ZERO for pretty much ever…

Makes me wonder why he's really doing this. Zuckerberg being Zuckerberg, it can't be out of any genuine sense of altruism. Probably just wants to crush all competitors before he monetizes the next generation of Meta AI.

He addresses this pretty clearly in the post. They don't want to be beholden to other companies to build the products they want to build. Their experience being under Apple's thumb on mobile strongly shaped this point of view.

Re: Open source AI is the path forward

#576

Post by nathansherburn » Tue, Jul 23, 2024, 11:53 PM UTC

sebastiennight wrote: post #525

I've summarized this entire thread in 4 lines (didn't even use AI for it!) Step 1. Chick-Fil-A releases a grass-fed beef burger to spite other fast-food joints, calls it "the vegan burger" Step 2. A couple of outraged vegans show up in the comments, pointing out that beef, even grass-fed beef, isn't vegan Step 3. Fast food enthusiasts push back: it's unreasonable to want companies to abide by this restrictive definit…

Open source software is one of our best and most passionately loved inventions. It'd be much easier to have a nuanced discussion about "open weights" but I don't think that's in Facebook's interest.

Re: Open source AI is the path forward

#577

Post by benreesman » Tue, Jul 23, 2024, 11:55 PM UTC

In general I look back on my time at FB with mixed feelings, I’m pretty skeptical that modern social media is a force for good and I was there early enough to have moved the needle.

But this is really positive stuff and it’s nice to view my time there through the lens of such a change for the better.

Keep up the good work on this folks.

Time to start thinking about opening up a little on the training data.

Re: Open source AI is the path forward

#578

Post by epolanski » Wed, Jul 24, 2024, 12:10 AM UTC

spullara wrote: post #300

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

It makes much more sense to invest in a next generation fab for GPUs than to buy GPUs and more closely matches this kind of project.

Does it? You're looking at a gargantuan investment in terms of money that would also require thousands of staff.

That just doesn't seem a good idea.

Re: Open source AI is the path forward

#579

Post by rkique » Wed, Jul 24, 2024, 12:28 AM UTC

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

Very much in this spirit is the NSF-funded National Deep Inference Fabric, which lets researchers run remote experiments on foundation models: https://ndif.us. They just announced a pilot program for Llama405b!

Re: Open source AI is the path forward

#580

Post by mupuff1234 » Wed, Jul 24, 2024, 12:29 AM UTC

whitepaint wrote: post #522

mupuff1234 wrote: post #318

Earlier quoted context omitted.

But you also know that the spider has been laying eggs so you better have an extermination plan ready.

Everyone is aware of that. No one thinks Facebook or Mark are some saint entities. But while the spider is doing some good deeds why not just go "yeah! go spider!". Once it becomes an asshole, we will kill it. People are not dumb.

It's not even truly open source, they set a user limit.