Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

451–460 of 936 posts

Re: Open source AI is the path forward

#451

Post by stavros »

_heimdall wrote: post #436
stavros wrote: post #425

Earlier quoted context omitted.

"Open weights" means you can use the weights for free (as in beer). "Open source" means you get the training dataset and the methodology. ~Nobody does open source LLMs.

Why is the dataset required for it to be open source? If I self host a project that is open sourced rather than paying for a hosted version, like Sentry.io for example, I don't expect data to come along with the code. Licensing rights are always up for debate in open source, but I wouldn't expect more than the code to be available and reviewable for anything needed to build and run the project. In the case of an LLM…

Data is to models what code is to software.

Re: Open source AI is the path forward

#452

Post by tambourine_man »

sheepscreek wrote: post #310
tambourine_man wrote: post #264

Earlier quoted context omitted.

Praising is good. Gratitude is a bit much. They got this big by selling user generated content and private info to the highest bidder. Often through questionable means. Also, the underdog always touts Open Source and standards, so it’s good to remain skeptical when/if tables turn.

All said and done, it is a very expensive and balsy way to undercut competitors. They’ve spent > $5B on hardware alone, much of which will depreciate in value quickly. Pretty sure the only reason Meta’s managed to do this is because of Zuck’s iron grip on the board (majority voting rights). This is great for Open Source and regular people though!

At Meta level, spending $5B to stay competitive is not balsy. It’s a bargain.

Re: Open source AI is the path forward

#453

Post by brrrrrm »

btbuildem wrote: post #271

The "open source" part sounds nice, though we all know there's nothing particularly open about the models (or their weights). The barriers to entry remain the same - huge upfront investments to train your own, and steep ongoing costs for "inference". Is the vision here to treat LLM-based AI as a "public good", akin to a utility provider in a civilized country (taxpayer funded, govt maintained, non-for-profit)? I thin…

check out the paper. it's pretty comprehensive https://ai.meta.com/research/publications/the-llama-3-herd-o...

Re: Open source AI is the path forward

#455

Post by mnahkies »

ygjb wrote: post #116
JumpCrisscross wrote: post #78

Earlier quoted context omitted.

> A GPU cluster might not be the correct investment, but a strategic investment in increasing, for example, the availability of training data, or interoperability of tools, or ease of use for building, training, and distributing models would probably pay big dividends Would you mind expanding on these options? Universal training data sounds intriguing.

Sure, just on the training front, building and maintaining a broad corpus of properly managed training data with metadata that provides attribution (for example, content that is known to be human generated instead of model generated, what the source of data is for datasets such as weather data, census data, etc), and that also captures any licensing encumbrance so that consumers of the training data can be confident…

I think the EPC (energy performance certificate) dataset in the UK is a nice example of this. Anyone can download a full dataset of EPC data from https://epc.opendatacommunities.org/

Admittedly it hasn't been cleaned all that much - you still need to put a bit of effort into that (newer certificates tend to be better quality), but it's very low friction overall. I'd love to see them do this with more datasets

Re: Open source AI is the path forward

#456

Post by gowld »

_heimdall wrote: post #436
stavros wrote: post #425

Earlier quoted context omitted.

"Open weights" means you can use the weights for free (as in beer). "Open source" means you get the training dataset and the methodology. ~Nobody does open source LLMs.

Why is the dataset required for it to be open source? If I self host a project that is open sourced rather than paying for a hosted version, like Sentry.io for example, I don't expect data to come along with the code. Licensing rights are always up for debate in open source, but I wouldn't expect more than the code to be available and reviewable for anything needed to build and run the project. In the case of an LLM…

https://opensource.org/osd

"The source code must be the preferred form in which a programmer would modify the program. Deliberately obfuscated source code is not allowed. Intermediate forms such as the output of a preprocessor or translator are not allowed."

> In the case of an LLM I would expect that to mean the code run to train the model, the code for the model data structure itself, and the control code for querying the model should all be available

The M in LLM is for "Model".

The code you describe is for an LLM harness, not for an LLM. The code for the LLM is whatever is needed to enable a developer to modify to inputs and then build a modified output LLM (minus standard generally available tools not custom-created for that product).

Training data is one way to provide this. Another way is some sort of semantic model editor for an interpretable model.

Re: Open source AI is the path forward

#458

Post by com2kid »

aiauthoritydev wrote: post #275
JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

Overall government doing anything is a bad idea. There are cases however where government is the only entity that can do certain things. These are things that involve military, law enforcement etc. Outside of this we should rely on private industry and for-profit industry as much as possible.

Ugh.

Government distorting undeveloped markets that have a lot of room for competition to increase efficiencies is a bad thing.

Government agencies running programs that should not be profitable, or where the only profit to be left comes at the expense of society as a whole, is a good thing.

Lots of basic medicine is the go to example here, treating cancer isn't going to be "profitable" and attempting to make it such just leads to dead people.

On the flip side, one can argue that dentistry has seen amazing strides in affordability and technological progress through the free market. From dental xrays to improvements in dental procedures to make them less painful for the patients.

Eye surgery is another area where competition has lead to good consumer outcomes.

But life of death situations where people can't spend time researching? The only profit there comes through exploiting people.

Re: Open source AI is the path forward

#459

Post by _heimdall »

solarmist wrote: post #446
_heimdall wrote: post #436

Earlier quoted context omitted.

Why is the dataset required for it to be open source? If I self host a project that is open sourced rather than paying for a hosted version, like Sentry.io for example, I don't expect data to come along with the code. Licensing rights are always up for debate in open source, but I wouldn't expect more than the code to be available and reviewable for anything needed to build and run the project. In the case of an LLM…

The sticking point is you can’t build the model. To be able to build the model from scratch you need methodology and a complete description of the data set. They only give you a blob of data you can run.

Got it, that makes sense. I still wouldn't expect them to have to publicly share the data itself, but if you can't take the code they share and run it against your own data to build a model that wouldn't be open source in my understanding of it.

Re: Open source AI is the path forward

#460

Post by gowld »

aerzen wrote: post #422
causal wrote: post #41

Earlier quoted context omitted.

"Open weights" is a more appropriate term but I'll point out that these weights are also largely inscrutable to the people with the code that trained it. And for licensing reasons, the datasets may not be possible to share. There is still a lot of modifying you can do with a set of weights, and they make great foundations for new stuff, but yeah we may never see a competitive model that's 100% buildable at home. Edit…

LLAMA is an open-weights model. I like this term, let's use that instead of open source.

Can a human programmer edit the weights according to some semantics?
Post reply on HN