Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

71–80 of 936 posts

Re: Open source AI is the path forward

#71

Post by exe34 »

JumpCrisscross wrote: post #38
candiddevmike wrote: post #32

Earlier quoted context omitted.

None of Meta's models are "open source" in the FOSS sense, even the latest Llama 3.1. The license is restrictive. And no one has bothered to release their training data either. This post is an ad and trying to paint these things as something they aren't.

> no one has bothered to release their training data If the FOSS community sets this as the benchmark for open source in respect of AI, they're going to lose control of the term. In most jurisdictions it would be illegal for the likes of Meta to release training data.

the training data is the source.

Re: Open source AI is the path forward

#72

Post by mesebrec »

redleader55 wrote: post #21
amusingimpala75 wrote: post #2

Sure but under what license? Because slapping “open source” on the model doesn’t make it open source if it’s not actually license that way. The 3.1 license still contains their non-commercial clause (over 700m users) and requires derivatives, whether fine tunings or trained on generated data, to use the llama name.

"Use it for whatever you want(conditions apply), but not if you are Google, Amazon, etc. If you become big enough talk to us." That's how I read the license, but obviously I might be missing some nuance.

You also can't use it for training or improving other models.

You also can't use it if you're the government of India.

Neither can sex workers use it. (Do you know if your customers are sex workers?)

There are also very vague restrictions for things like discrimination, racism etc.

Re: Open source AI is the path forward

#73

Post by mkolodny »

the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

Llama’s code is open source: https://github.com/meta-llama/llama3/blob/main/llama/model.p...

Re: Open source AI is the path forward

#74

Post by cs702 »

> We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models.

Bravo! While I don't agree with Zuck's views and actions on many fronts, on this occasion I think he and the AI folks at Meta deserve our praise and gratitude. With this release, they have brought the cost of pretraining a frontier 400B+ parameter model to ZERO for pretty much everyone -- well, everyone except Meta's key competitors.[a] THANK YOU ZUCK.

Meanwhile, the business-minded people at Meta surely won't mind if the release of these frontier models to the public happens to completely mess up the AI plans of competitors like OpenAI/Microsoft, Google, Anthropic, etc. Come to think of it, the negative impact on such competitors was likely a key motivation for releasing the new models.

---

[a] The license is not open to the handful of companies worldwide which have more than 700M users.

Re: Open source AI is the path forward

#75

Post by the8thbit »

bilsbie wrote: post #26
the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

Can’t you do fine tuning on those binaries? That’s a modification.

You can fine tune the models, and you can modify binaries. However, there is no human readable "source" to open in either case. The act of "fine tuning" is essentially brute forcing the system to gradually alter the weights such that loss is reduced against a new training set. This limits what you can actually do with the model vs an actual open source system where you can understand how the system is working and modify specific functionality.

Additionally, models can be (and are) fine tuned via APIs, so if that is the threshold required for a system to be "open source", then that would also make the GPT4 family and other such API only models which allow finetuning open source.

Re: Open source AI is the path forward

#77

Post by blackeyeblitzar »

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

What about distributed training on volunteer hardware? Is that feasible?

Re: Open source AI is the path forward

#78

Post by JumpCrisscross »

ygjb wrote: post #64
CardenB wrote: post #22

Earlier quoted context omitted.

Doubtful that GPUs purchased today would be in use for a similar time scale. Govt investment would also drive the cost of GPUs up a great deal. Not sure why a publicly accessible GPU cluster would be a better solution than the current system of research grants.

Of course they won't. The investment in the Heavy Press Program was the initial build, and just citing one example, the Alcoa 50,000 ton forging press was built in 1955, operated until 2008, and needed ~$100M to get it operational again in 2012. The investment was made to build the press, which created significant jobs and capital investment. The press, and others like it, were subsequently operated by and then sold…

> A GPU cluster might not be the correct investment, but a strategic investment in increasing, for example, the availability of training data, or interoperability of tools, or ease of use for building, training, and distributing models would probably pay big dividends

Would you mind expanding on these options? Universal training data sounds intriguing.

Re: Open source AI is the path forward

#79

Post by gpm »

JumpCrisscross wrote: post #45
rvnx wrote: post #43

Earlier quoted context omitted.

It would be better that the government removes IP on such technology for public use, like drugs got generics. This way the government pays 2'500 USD per card, not 40'000 USD or whatever absurd.

> better that the government removes IP on such technology for public use, like drugs got generics You want to punish NVIDIA for calling its shots correctly? You don't see the many ways that backfires?

No. But I do want to limit the amount we reward NVIDIA for calling the shots correctly to maximize the benefit to society. For instance by reducing the duration of the government granted monopolies on chip technology that is obsolete well before the default duration of 20 years is over.

That said, it strikes me that the actual limiting factor is fab capacity not nvidia's designs and we probably need to lift the monopolies preventing competition there if we want to reduce prices.

Re: Open source AI is the path forward

#80

Post by apsec112 »

mkolodny wrote: post #73
the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

Llama’s code is open source: https://github.com/meta-llama/llama3/blob/main/llama/model.p...

That's not the training code, just the inference code. The training code, running on thousands of high-end H100 servers, is surely much more complex. They also don't open-source the dataset, or the code they used for data scraping/filtering/etc.
Post reply on HN