Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

521–530 of 936 posts

Re: Open source AI is the path forward

#521

Post by randomdata »

ygjb wrote: post #116
JumpCrisscross wrote: post #78

Earlier quoted context omitted.

> A GPU cluster might not be the correct investment, but a strategic investment in increasing, for example, the availability of training data, or interoperability of tools, or ease of use for building, training, and distributing models would probably pay big dividends Would you mind expanding on these options? Universal training data sounds intriguing.

Sure, just on the training front, building and maintaining a broad corpus of properly managed training data with metadata that provides attribution (for example, content that is known to be human generated instead of model generated, what the source of data is for datasets such as weather data, census data, etc), and that also captures any licensing encumbrance so that consumers of the training data can be confident…

If the public is going to go to all the trouble of doing something, why would that public not make it clear that there is no legal threat to using any data available?

The public is incredibly lazy, though. Don't expect them to do anything until their hand is forced, which doesn't bode well for the action to meet a desirable outcome.

Re: Open source AI is the path forward

#522

Post by whitepaint »

mupuff1234 wrote: post #318
yesco wrote: post #256

Earlier quoted context omitted.

Does it really matter? Attributing goodwill to a company is like attributing goodwill to a spider that happens to clean up the bugs in your basement. Sure if they had the ability to, I'm confident Meta would try something like that, but they obviously don't, and will not for the foreseeable future. I have faith they will continue to do what's in their best interests and if their best interests happen to align with mi…

But you also know that the spider has been laying eggs so you better have an extermination plan ready.

Everyone is aware of that. No one thinks Facebook or Mark are some saint entities. But while the spider is doing some good deeds why not just go "yeah! go spider!". Once it becomes an asshole, we will kill it. People are not dumb.

Re: Open source AI is the path forward

#524

Post by yjftsjthsd-h »

macrolime wrote: post #427
yjftsjthsd-h wrote: post #416

Earlier quoted context omitted.

> The file I linked to, model.py, contains the Llama 3 model itself. That makes it source available ( https://en.wikipedia.org/wiki/Source-available_software ), not open source

Source available means you can see the source, but not modify it. This is kinda the opposite, you can modify the model, but you don't see all the details of its creation.

> Source available means you can see the source, but not modify it.

No, it doesn't mean that. To quote the page I linked, emphasis mine,

> Source-available software is software released through a source code distribution model that includes arrangements where the source can be viewed, and in some cases modified, but without necessarily meeting the criteria to be called open-source. The licenses associated with the offerings range from allowing code to be viewed for reference to allowing code to be modified and redistributed for both commercial and non-commercial purposes.

> This is kinda the opposite, you can modify the model, but you don't see all the details of its creation.

Per https://github.com/meta-llama/llama3/blob/main/LICENSE there's also a laundry list of ways you're not allowed to use it, including restrictions on commercial use. So not Open Source.

Re: Open source AI is the path forward

#525

Post by sebastiennight »

I've summarized this entire thread in 4 lines (didn't even use AI for it!)

Step 1. Chick-Fil-A releases a grass-fed beef burger to spite other fast-food joints, calls it "the vegan burger"

Step 2. A couple of outraged vegans show up in the comments, pointing out that beef, even grass-fed beef, isn't vegan

Step 3. Fast food enthusiasts push back: it's unreasonable to want companies to abide by this restrictive definition of "vegan". Clearly this burger is a gamechanger and the definition needs to adapt to the times.

Step 4. Goto Step 2 in an infinite loop

Re: Open source AI is the path forward

#526

Post by meowtimemania »

war321 wrote: post #450

Even if it's just open weights and not "true" open source, I'll still give Meta the appreciation of being one of the few big AI companies actually committed to open models. In an ecosystem where groups like Anthropic and OpenAI keep hemming and hawing about safety and the necessity of closed AI systems "for our sake", they stand out among the rest.

Why would openai/anthropic's approach be more safe? Are people able to remove all the guard rails on the llama models?

Re: Open source AI is the path forward

#527

Post by modo_mario »

bun_at_work wrote: post #121

Meta makes their money off advertising, which means they profit from attention. This means they need content that will grab attention, and creating open source models that allow anyone to create any content on their own becomes good for Meta. The users of the models can post it to their Instagram/FB/Threads account. Releasing an open model also releases Meta from the burden of having to police the content the model g…

I think the biggest part of it is just that they were behind but also betting on it. This allowed them to get a lot of traction, support and be a notable player in the race whilst still retaining some control. Chances are if someone is going to have a frontrow seat monetizing this it's still them.

Re: Open source AI is the path forward

#528

Post by nine_k »

stavros wrote: post #425
causal wrote: post #41

Earlier quoted context omitted.

"Open weights" is a more appropriate term but I'll point out that these weights are also largely inscrutable to the people with the code that trained it. And for licensing reasons, the datasets may not be possible to share. There is still a lot of modifying you can do with a set of weights, and they make great foundations for new stuff, but yeah we may never see a competitive model that's 100% buildable at home. Edit…

"Open weights" means you can use the weights for free (as in beer). "Open source" means you get the training dataset and the methodology. ~Nobody does open source LLMs.

Linux is open source and is mostly C code. You cannot run C code directly, you have to compile it and produce binaries. But it's the C code, not binary form, where the collaboration happens.

With LLMs, weights are the binary code: it's how you run the model. But to be able to train the model from scratch, or to collaborate on new approaches, you have to operate at a the level of architecture, methods, and training data sets. They are the source code.

Re: Open source AI is the path forward

#529

Post by kiwih »

xena wrote: post #352
jszymborski wrote: post #296

Earlier quoted context omitted.

As you've rightly pointed out, we have the mechanism, now let's fund it properly! I'm in Canada, and our science funding has likewise fallen year after year as a proportion of our GDP. I'm still benefiting from A100 clusters funded by tax payer dollars, but think of the advantage we'd have over industry if we didn't have to fight over resources.

Where do you get access to those as a member of the general public?

In Australia at least, anyone who is enrolled at or works at a university can use the taxpayer-subsidised "Gadi" HPC which is part of the National Computing Infrastructure (https://nci.org.au/our-systems/hpc-systems). I also do mean anyone, I have an undergraduate student using it right now (for free) to fine-tune several LLMs.

It also says commercial orgs can get access via negotiation, I expect a random member of the public would be able to go that route as well. I expect that there would be some hurdles to cross, it isn't really common for random members of the public to be doing the kinds of research Gadi was created to benefit. I expect it is the same way in this case in Canada. I suppose the argument is if there weren't any gatekeeping at all, you might end up with all kinds of unsuitable stuff on the cluster, e.g. crypto miners and such.

Possibly another way for a true random person to get access would be to get some kind of 0-hour academic affiliation via someone willing to back you up, or one could enrol in a random AI course or something and then talk to the lecturer in charge.

In reality, the (also taxpayer-subsidised) university pays some fee for access, but it doesn't come from any of our budgets.

Re: Open source AI is the path forward

#530

Post by loceng »

war321 wrote: post #450

Even if it's just open weights and not "true" open source, I'll still give Meta the appreciation of being one of the few big AI companies actually committed to open models. In an ecosystem where groups like Anthropic and OpenAI keep hemming and hawing about safety and the necessity of closed AI systems "for our sake", they stand out among the rest.

To me it will be most interesting to see who attempts to manipulate the models by stuffing them with content, essentially adding "duplicate" content such as via tautology, in order to make it have added-misallocated weight; which I don't think an AI model will automatically be able to determine, unless it was truly intelligent, instead it would require to be trained by competent humans.

And so the models that have mechanisms for curating and preventing such misapplied weighting, and then the organizations and individuals who accurately create adjustments to the models, will in the end be the winners - where truth has been more honed for.

Post reply on HN