Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

491–500 of 936 posts

Re: Open source AI is the path forward

#491

Post by GuB-42 »

the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

I like the term "open weights". Open source would be the dataset and code that generates these weights.

There is still a lot you can do with weights, like fine tuning, and it is arguably more useful as retraining the entire model would cost millions in compute.

Re: Open source AI is the path forward

#492

Post by ab5tract »

gowld wrote: post #469
downWidOutaFite wrote: post #384

Earlier quoted context omitted.

OSI made themselves the authority because they hated Richard Stallman and his Free Software movement. It's just marketing.

RMS has no interest in governing Open Source, so your comment bears no particular relevance. RMS is an advocate for Free Software. Free Software generally implies Open Source, but not the converse. RMS considers openness of source to be a separate category from the freeness of software. "Free software is a political movement; open source is a development model." https://www.gnu.org/licenses/license-list.en.html

Are you really pretending that OSI and the open source label itself wasn’t a reactionary movement that vilified free software principles in hopes of gaining corporate traction?

Most of us who were there remember it differently. True open source advocates will find little to refute in what I’ve said.

Re: Open source AI is the path forward

#493

Post by jpadkins »

the8thbit wrote: post #75
bilsbie wrote: post #26

Earlier quoted context omitted.

Can’t you do fine tuning on those binaries? That’s a modification.

You can fine tune the models, and you can modify binaries. However, there is no human readable "source" to open in either case. The act of "fine tuning" is essentially brute forcing the system to gradually alter the weights such that loss is reduced against a new training set. This limits what you can actually do with the model vs an actual open source system where you can understand how the system is working and mod…

> vs an actual open source system where you can understand how the system is working and modify specific functionality.

No one on the planet understands how the model weights work exactly, nor can they modify them specifically (i.e. hand modifying the weights to get the result they want). This is an impossible standard.

The source code is open (sorta, it does have some restrictions). The weights are open. The training data is closed.

Re: Open source AI is the path forward

#494

Post by whartung »

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

Now, I have no idea.

How much capability would $3.2bn in terms of AI computing power provide, including the operational and power costs of the cluster?

Certainly, you could build a "$3.2bn GPU cluster", but it would be dark.

So, how much learning time would $3.2bn provide? 1 year? 10 years?

Just curious about hand wavy guesses. I have no idea the scope of the these clusters.

Re: Open source AI is the path forward

#495

Post by _heimdall »

gowld wrote: post #456
_heimdall wrote: post #436

Earlier quoted context omitted.

Why is the dataset required for it to be open source? If I self host a project that is open sourced rather than paying for a hosted version, like Sentry.io for example, I don't expect data to come along with the code. Licensing rights are always up for debate in open source, but I wouldn't expect more than the code to be available and reviewable for anything needed to build and run the project. In the case of an LLM…

https://opensource.org/osd "The source code must be the preferred form in which a programmer would modify the program. Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a preprocessor or translator are not allowed." > In the case of an LLM I would expect that to mean the code run to train the model, the code for the model data structure itself, and the control code for query…

I still don't quite follow. If Meta were to provide all code required to train a model (it sounds like they don't), and they provided the code needed to query the model you train to get answers how is that not open source?

> Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a preprocessor or translator are not allowed.

This definition actually makes it impossible for any LLM to be considered open source until the interpretability problem is solved. The trained model is functionally obfuscated code, it can't be read or interpreted by a human.

We may be saying the same thing here, I'm not quite sure if you're saying the model must be available or if what is missing is the code to train your own model.

Re: Open source AI is the path forward

#496

Post by visarga »

bun_at_work wrote: post #121

Meta makes their money off advertising, which means they profit from attention. This means they need content that will grab attention, and creating open source models that allow anyone to create any content on their own becomes good for Meta. The users of the models can post it to their Instagram/FB/Threads account. Releasing an open model also releases Meta from the burden of having to police the content the model g…

> Meta makes their money off advertising, which means they profit from attention. This means they need content that will grab attention

That is why they hopped on the Attention is All You Need train

Re: Open source AI is the path forward

#497

Post by ori_b »

celdon25 wrote: post #396
quasse wrote: post #355

Earlier quoted context omitted.

I'm just a lowly outsider to the AI space, but calling these open source models seems kind of like calling a compiled binary open source. If you don't have a way to replicate what they did to create the model, it seems more like freeware than open source.

> it seems more like freeware than open source. What would you have them do instead? Specifically?

Release the training set and the code that was used to train the model, or stop calling it open source.

If you can't fork it and take the project in your own direction, it's not open source.

Re: Open source AI is the path forward

#498

Post by actinium226 »

gorgoiler wrote: post #462
the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

One counterpoint is that major publications (eg New York Times) would have you believe that AI is a mildly lossy compression algorithm capable of reconstructing the original source material.

It's not?

Re: Open source AI is the path forward

#499

Post by tshaddox »

pphysch wrote: post #488
thayne wrote: post #374

Earlier quoted context omitted.

I think it would also include the code used to train it

That would be more analogous to the build toolchain than the source code, but yes

Surely traditional “open source” also needs some notion of a reproducible build toolchain, otherwise the source code itself is approximately useless.

Imagine if the source code was in a programming language of which the basic syntax and semantics were known to no one but the original developers.

Or more realistically, I think it’s a major problem if an open source project can only be built by an esoteric process that only the original developers have access to.

Re: Open source AI is the path forward

#500

Post by szundi »

the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

Open source = reproducible binaries (weights) by you on your computer, IMO.

Strategy of FB is that they are good to be a user only and fine ruining competitor’s business with good enough free alternatives while collecting awards as saviors of whatever.

Post reply on HN