Open source AI is the path forward • Hacker News Board

Re: Open source AI is the path forward

#141

Post by whimsicalism » Tue, Jul 23, 2024, 4:23 PM UTC

abetusk wrote: post #27

Another case of "open-washing". Llama is not available open source, under the common definition of open source, as the license doesn't allow for commercial re-use by default [0]. They provide their model, with weights and code, as "source available" and it looks like they allow for commercial use until a 700M monthly subscriber cap is surpassed. They also don't allow you to train other AI models with their model: """…

i think these clauses are unenforceable. it's telling that OAI hasn't tried a similar suit despite multiple extremely well-known cases of competitors training on OAI outputs

Re: Open source AI is the path forward

#142

Post by enriquto » Tue, Jul 23, 2024, 4:23 PM UTC

It's alarming that he refers to llama as if it was open source.

The definition of free software (and open source, for that mater), is well-established. The same definition applies to all programs, whether they are "AI" or not. In any case, if a program was built by training against a dataset, the whole dataset is part of the source code.

Llama is distributed in binary form, and it was built based on a secret dataset. Referring to it as "open source" is not ignorance, it's malice.

Re: Open source AI is the path forward

#143

Post by diggan » Tue, Jul 23, 2024, 4:23 PM UTC

JumpCrisscross wrote: post #113

diggan wrote: post #107

Earlier quoted context omitted.

So because it's really hard to do proper Open Source with these LLMs, means we need to change the meaning of Open Source so it fits with these PR releases?

> because it's really hard to do proper Open Source with these LLMs, means we need to change the meaning of Open Source so it fits with these PR releases? Open training data is hard to the point of impracticality. It requires excluding private and proprietary data. Meanwhile, the term "open source" is massively popular. So it will get used. The question is how. Meta et al would love for the choice to be between, on o…

> Open training data is hard to the point of impracticality. It requires excluding private and proprietary data.

Right, so the onus is on Facebook/Meta to get that right, then they could call something Open Source, until then, find another name that already doesn't have a specific meaning.

> (The way open source software, today, generally means source available, not FOSS.)

No, but it's going in that way. Open Source, today, still means that the things you need to build a project, is publicly available for you to download and run on your own machine, granted you have the means to do so. What you're thinking of is literally called "Source Available" which is very different from "Open Source".

The intent of Open Source is for people to be able to reproduce the work themselves, with modifications if they want to. Is that something you can do today with the various Llama models? No, because one core part of the projects "source code" (what you need to reproduce it from scratch), the training data, is being held back and kept private.

Re: Open source AI is the path forward

#144

Post by the8thbit » Tue, Jul 23, 2024, 4:23 PM UTC

stale2002 wrote: post #82

the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

Ok call it Open Weights then if the dictionary definitions matter so much to you. The actual point that matters is that these models are available for most people to use for a lot of stuff, and this is way way better than what competitors like OpenAI offer.

They don't "[allow] developers to modify its code however they want", which is a critical component of "open source", and one that Meta is clearly trying to leverage in branding around its products. I would like them to start calling these "public weight models", because what they're doing now is muddying the waters so much that "open source" now just means providing an enormous binary and an open source harness to run it in, rather than serving access to the same binary via an API.

Re: Open source AI is the path forward

#145

Post by [deleted] » Tue, Jul 23, 2024, 4:24 PM UTC

[deleted]

Re: Open source AI is the path forward

#146

Post by oersted » Tue, Jul 23, 2024, 4:24 PM UTC

blackeyeblitzar wrote: post #77

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

What about distributed training on volunteer hardware? Is that feasible?

It is an exciting concept, there's a huge wealth of gaming hardware deployed that is inactive at most hours of the day. And I'm sure people are willing to pay well above the electricity cost for it.

Unfortunately, the dominant LLM architecture makes it relatively infeasible right now.

- Gaming hardware has too limited VRAM for training any kind of near-state-of-the-art model. Nvidia is being annoyingly smart about this to sell enterprise GPUs at exorbitant markups.

- Right now communication between machines seems to be the bottleneck, and this is way worse with limited VRAM. Even with data-centre-grade interconnect (mostly Infiniband, which is also Nvidia, smart-asses), any failed links tend to cause big delays in training.

Nevertheless, it is a good direction to push towards, and the government could indeed help, but it will take time. We need both a more healthy competitive landscape in hardware, and research towards model architectures that are easy to train in a distributed manner (this was also the key to the success of Transformers, but we need to go further).

Re: Open source AI is the path forward

#147

Post by whimsicalism » Tue, Jul 23, 2024, 4:25 PM UTC

ks2048 wrote: post #91

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

How about using some of that money to develop CUDA alternatives so everyone is not paying the Nvidia tax?

It seems like rocm is already fully ready for transformer inference, so you are just referring to training?

Re: Open source AI is the path forward

#148

Post by rvnx » Tue, Jul 23, 2024, 4:25 PM UTC

Teever wrote: post #88

JumpCrisscross wrote: post #45

Earlier quoted context omitted.

> better that the government removes IP on such technology for public use, like drugs got generics You want to punish NVIDIA for calling its shots correctly? You don't see the many ways that backfires?

There was a post[0] on here recently about how the US went from producing woefully insufficient numbers of aircraft to producing 300k by the end of world war 2. One of the things that the post mentioned was the meager profit margin that the companies made during this time. But the thing is that this set the America auto and aviation industry up to rule the world for decades. A government going to a company and saying…

In the case of NVIDIA it's even more sneaky.

They are an intellectual property company holding the rights on plans to make graphic cards, not even a company actually making graphic cards.

The government could launch an initiative "OpenGPU" or "OpenAI Accelerator", where the government orders GPUs from TSMC directly, without the middleman.

It may require some tweaking in the law to allow exception to intellectual property for "public interest".

Re: Open source AI is the path forward

#149

Post by indus » Tue, Jul 23, 2024, 4:25 PM UTC

Is there an argument against Open Source AI?

Not the usual nation-state rhetoric, but something that justifies that closed source leads to better user-experience and fewer security and privacy issues.

An ecosystem that benefits vendors, customers, and the makers of close source?

Are there historical analogies other than Microsoft Windows or Apple iPhone / iOS?

Re: Open source AI is the path forward

#150

Post by JumpCrisscross » Tue, Jul 23, 2024, 4:26 PM UTC

plsbenice34 wrote: post #124

JumpCrisscross wrote: post #84

Earlier quoted context omitted.

> the training data is the source Sure. But that's not going to be released. The term open source AI cannot be expected to cover it because it's not practical.

Of course it could be practical - provide the data. The fact of that society is a dystopian nightmare controlled by a few megacorporations that don't want free information does not justify outright changing the meaning of the language.

> provide the data

Who? It's not their data.