Open source AI is the path forward • Hacker News Board

Re: Open source AI is the path forward

#271

Post by btbuildem » Tue, Jul 23, 2024, 5:20 PM UTC

The "open source" part sounds nice, though we all know there's nothing particularly open about the models (or their weights). The barriers to entry remain the same - huge upfront investments to train your own, and steep ongoing costs for "inference".

Is the vision here to treat LLM-based AI as a "public good", akin to a utility provider in a civilized country (taxpayer funded, govt maintained, non-for-profit)?

I think we could arguably call this "open source" when all the infra blueprints, scripts and configs are freely available for anyone to try and duplicate the state-of-the-art (resource and grokking requirements nonwithstanding)

Re: Open source AI is the path forward

#272

Post by vorticalbox » Tue, Jul 23, 2024, 5:21 PM UTC

throwaway_2494 wrote: post #241

cs702 wrote: post #74

> We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models. Bravo! While I don't agree with Zuck's views and actions on many fronts, on this occasion I think he and the AI folks at Meta deserve our praise and gratitude. With this release, they have brought the cost of pretraining a frontier 400B+ parameter model to ZERO for pretty much ever…

> We’re releasing Llama 3.1 405B Is it possible to run this with ollama?

If you have the ram for it.

Ollama will offload as many layers as it can to the gpu then the rest will run on the cpu/ram.

Re: Open source AI is the path forward

#273

Post by sailingparrot » Tue, Jul 23, 2024, 5:21 PM UTC

the8thbit wrote: post #144

stale2002 wrote: post #82

Earlier quoted context omitted.

Ok call it Open Weights then if the dictionary definitions matter so much to you. The actual point that matters is that these models are available for most people to use for a lot of stuff, and this is way way better than what competitors like OpenAI offer.

They don't "[allow] developers to modify its code however they want", which is a critical component of "open source", and one that Meta is clearly trying to leverage in branding around its products. I would like them to start calling these "public weight models", because what they're doing now is muddying the waters so much that "open source" now just means providing an enormous binary and an open source harness to r…

Feels a bit like you are splitting hair for the pleasure of semantic arguments to be honest. Yes there are no source in ML, so if we want to be pedantic it shouldn't be called open source. But what really matters in the open source movement is that we are able to take a program built by someone and modify it to do whatever we want with it, without having to ask someone for permission or get scrutinized or have to pay someone.

The same applies here, you can take those models and modify them to do whatever you want (provided you know how to train ML models), without having to ask for permission, get scrutinized or pay someone.

I personally think using the term open source is fine, as it conveys the intent correctly, even if, yes, weights are not sources you can read with your eyes.

Re: Open source AI is the path forward

#274

Post by openrisk » Tue, Jul 23, 2024, 5:22 PM UTC

Open source "AI" is a proxy for democratising and making (much) more widely useful the goodies of high performance computing (HPC).

The HPC domain (data and compute intensive applications that typically need vector, parallel or other such architectures) have been around for the longest time, but confined to academic / government tasks.

LLM's with their famous "matrix multiply" at their very core are basically demolishing an ossified frontier where a few commercial entities (Intel, Microsoft, Apple, Google, Samsung etc) have defined for decades what computing looks like for most people.

Assuming that the genie is out of the bottle, the question is: what is the shape of end-user devices that are optimally designed to use compute intensive open source algorithms? The "AI PC" is already a marketing gimmick, but could it be that Linux desktops and smartphones will suddenly be "ΑΙ natives"?

For sure its a transformational period and the landscape T+10 yrs could be drastically different...

Re: Open source AI is the path forward

#275

Post by aiauthoritydev » Tue, Jul 23, 2024, 5:24 PM UTC

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

Overall government doing anything is a bad idea. There are cases however where government is the only entity that can do certain things. These are things that involve military, law enforcement etc. Outside of this we should rely on private industry and for-profit industry as much as possible.

Re: Open source AI is the path forward

#276

Post by erickj » Tue, Jul 23, 2024, 5:26 PM UTC

ks2048 wrote: post #91

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

How about using some of that money to develop CUDA alternatives so everyone is not paying the Nvidia tax?

That's the kind of work that can come out of academia and open source communities when societies provide the resources required.

Re: Open source AI is the path forward

#277

Post by emporas » Tue, Jul 23, 2024, 5:26 PM UTC

the8thbit wrote: post #161

whimsicalism wrote: post #122

Earlier quoted context omitted.

I don't find this argument super convincing. There's a pretty clear difference between the 'finetuning' offered via API by GPT4 and the ability to do whatever sort of finetuning you want and get the weights at the end that you can do with open weights models. "Brute forcing" is not the correct language to use for describing fine-tuning. It is not as if you are trying weights randomly and seeing which ones work on you…

"There's a pretty clear difference between the 'finetuning' offered via API by GPT4 and the ability to do whatever sort of finetuning you want and get the weights at the end that you can do with open weights models." Yes, the difference is that one is provided over a remote API, and the provider of the API can restrict how you interact with it, while the other is performed directly by the user. One is a SaaS solution…

> When I modify source code, I might make a change, check what that does, change the same functionality again, check the new change, etc... up to maybe a couple dozen times.

You can modify individual neurons if you are so inclined. That's what Anthropic have done with the Claude family of models [1]. You cannot do that using any closed model. So "Open Weights" looks very much like "Open Source".

Techniques for introspection of weights are very primitive, but i do think new techniques will be developed, or even new architectures which will make it much easier.

[1] https://www.anthropic.com/news/mapping-mind-language-model

Re: Open source AI is the path forward

#278

Post by pavlov » Tue, Jul 23, 2024, 5:26 PM UTC

aiauthoritydev wrote: post #275

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

Overall government doing anything is a bad idea. There are cases however where government is the only entity that can do certain things. These are things that involve military, law enforcement etc. Outside of this we should rely on private industry and for-profit industry as much as possible.

The American healthcare industry demonstrates the tremendous benefits of rigidly applying this mindset.

Why couldn’t law enforcement be private too? You call 911, several private security squads rush to solve your immediate crime issue, and the ones who manage to shoot the suspect send you a $20k bill. Seems efficient. If you don’t like the size of the bill, you can always get private crime insurance.

Re: Open source AI is the path forward

#279

Post by skybrian » Tue, Jul 23, 2024, 5:26 PM UTC

Yizahi wrote: post #132

skybrian wrote: post #61

Earlier quoted context omitted.

I think you’re right from the perspective of an individual developer. You and I are not about to fork Chromium any time soon. If you presume that forking is impractical then sure, the right to fork isn’t worth much. But just because a single developer couldn’t do it doesn’t mean it couldn’t be done. It means nobody has organized a large enough effort yet. For something like a browser, which is critical for security,…

If Microsoft can't do it, then we can reasonably conclude that it can't be done for any practical purpose. Discussing infinitesimal possibilities is better left to philosophers.

Doesn’t Microsoft maintain its own fork of Chromimum?

Re: Open source AI is the path forward

#280

Post by JumpCrisscross » Tue, Jul 23, 2024, 5:26 PM UTC

diggan wrote: post #231

JumpCrisscross wrote: post #222

Earlier quoted context omitted.

> For the CH Open, I'm not finding anything specific, even from Swiss websites, could you help me understand what you're referring to here Was on the HN front page earlier [1][2]. The definition comes strikingly close to source on request with no use restrictions. > all these definitions have at least some points in common Agreed. But they're all different. There isn't an accepted defintiion of open source even when…

> Agreed. But they're all different. There isn't an accepted defintiion of open source even when it comes to software; there is an accepted set of broad principles. Agreed, but are we splitting hairs here and is it relevant to the claim made earlier? > (The way open source software, today, generally means source available, not FOSS.) Do any of these principles or definitions from these orgs agree/disagree with that?…

> are we splitting hairs here and is it relevant to the claim made earlier?

I don't think so. Take the Swiss definition. Source on request, not even available. Yet being branded and accepted as open source.

(To be clear, the Swiss example favours FOSS. But it also permits source on request and bundles them together under the same label.)