Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta to release open-source commercial AI model

zdnet.com

Post reply on HN
Print view View original

141–150 of 168 posts

Re: Meta to release open-source commercial AI model

#141

Post by pmarreck »

freedomben wrote: post #127
pmarreck wrote: post #66

I have a 128 core Threadripper, a 2080 Ti and a 3080 Ti. How can I play with open source LLM's locally?

May I ask why you have such an amazing machine, and two nice graphics cards? Feel free to tell me it's none of my business, it's just very interesting to me :-)

Career dev who had the cash and wanted to experiment with anything that can be done concurrently, such as in my language of choice lately, which features high concurrency (https://elixir-lang.org/) or these LLM's, or anything else that can be done in massively parallel fashion (which is, perhaps surprisingly, only a minority of possible computer work, but it still means I can run many apps without much slowdown!)

I originally had 2 2080ti's to experiment also with virtio/proxmox (you need 1 for the host and 1 for any VM you run). I never got that running successfully at the time, but then Proton got really good (I mainly just wanted to run windows games fast in a VM, but that circumvented that). Later on I upgraded one of them to a 3080ti.

It's a System76 machine, they make good stuff

Re: Meta to release open-source commercial AI model

#142

Post by brucethemoose2 »

Auracle wrote: post #137
Der_Einzige wrote: post #107

Earlier quoted context omitted.

It’s trivial to use these methods to produce real looking images, or even stuff in the likeness of real people…

Yeah. I did a fine tuned model of my daughter and niece and I definitely have to put in “sexy, naked,” and the like in the negative prompt when using them. I don’t think society is going to have a hissyfit until some app comes along that makes it super easy for people to train good models locally on people and then generate whatever they want. That day’s coming really soon though.

There are tons of web services for this. They are just obscure and distributed enough to avoid public ire.

The pieces to do local LORA training are all there, but honestly the tyranny of CUDA is the biggest blocker for the average person.

Re: Meta to release open-source commercial AI model

#143

Post by brucethemoose2 »

bilsbie wrote: post #140

Is it possible to do further training on the weights they release?

Yes, and there are a sea of finetunes. See: https://huggingface.co/models?sort=modified&search=Ggml

QLORA is the most cost effective method so far. Some people also do finetuning on Google TPUs

Re: Meta to release open-source commercial AI model

#144

Post by sifar »

sagebird wrote: post #132

repeat after me: hardware is the only moat If you want to live the good life before you are exquisitely extinguished, spend every other day figuring out how to buy more NVDA, the other days exercising outside, being human.

until better algorithms or newer paradigm obviate the need for large memory/computations

Re: Meta to release open-source commercial AI model

#145

Post by brucethemoose2 »

whimsicalism wrote: post #90

If you read past the title, this article is not at all clear if they are referring to a commercial offering (ie. license our model for $$) or an open-source license with commercial usage (Apache, etc.) My guess is still the latter because that's what I've heard the rumors about, but this article is pretty unclear on this fact.

Falcon 40B was released as a "free with royalties above a certain amount of profit" license, and got roasted for it. It was so bad that they changed the license to Apache.

I don't think any business would run such a "licensed" model over MPT 30B or Falcon 40B, unless its way better than LLaMA 65b.

Re: Meta to release open-source commercial AI model

#147

Post by IshKebab »

l33t233372 wrote: post #139
whimsicalism wrote: post #82

Earlier quoted context omitted.

Strong disagree - I think OSS is fine framing of this. Weights are a third category, you can 'fork' them in an a way that you can't with standard binaries.

You can add hooks to functions and “fork” binaries, which is a pretty similar effort to adding training data to given model weights.

Nobody does that because if you only have binaries you probably don't have permission to do that. Plus it's impractical to make any significant changes that way.

Re: Meta to release open-source commercial AI model

#148

Post by IshKebab »

TX81Z wrote: post #71
ramshanker wrote: post #67

Earlier quoted context omitted.

Sometimes, I wonder what if someone in XYZ country downloads whole of Z-Library/Libgen, all the books ever printed, and all the papers ever published, all the newspapers and so on. and releases the model open source. There are jurisdictions with Lax rules. And they will have much better knowledge, answers, etc than the western, Lawyer approved models. Sometimes knowledge needs to be set free I guess.

The production of knowledge needs to be funded as it isn’t “free”. Copyright and licensing is one model that has worked for a long time. It has flaws, but it has produced good things. At this point with the quality of current web content and the collapse of journalism as an industry I think we can say online ads have utterly failed as a replacement income stream. Unless you want all LLM to say “I’m sorry the data I w…

The production of knowledge (I assume you're mainly talking about scientific research here) is absolutely not funded by copyright royalties or anything like that.

Journals get their content for free. Actually often they charge the authors for it.

Research is mainly funded by governments and taxes.

Re: Meta to release open-source commercial AI model

#149

Post by l33t233372 »

roody15 wrote: post #64
strikelaserclaw wrote: post #55

Earlier quoted context omitted.

kind of a dystopian nightmare world in which large corporations utilize AI to create low cost, infinite content that humans engage with (mostly content catering to the human tendency for tribalism, prestige, sexual desires etc...), sounds like we are creating a world similar to the Matrix.

I think we may have already entered it. Infinite scroll based feeds like TikTok, Instagram, and Threads (and possible Reddit these days) … just AI algorithm deciding what you should find “entertaining” or “important”. It’s really the ultimate nightmare with the internet becoming just TV 3.0 in which content is controlled and curated … you just consume mindlessly. Any attempts to create a Reddit clone.. or system in w…

I agree with the sentiment, but I disagree with this:

> Any attempts to create a Reddit clone.. or system in which people freely communicate is now “regulated” for “hate” speech or “terrorism”.

The fact is that many “systems in which people freely communicate” are regularly first adopted by people who participate in hate speech and terrorism.

Re: Meta to release open-source commercial AI model

#150

Post by whimsicalism »

brucethemoose2 wrote: post #145
whimsicalism wrote: post #90

If you read past the title, this article is not at all clear if they are referring to a commercial offering (ie. license our model for $$) or an open-source license with commercial usage (Apache, etc.) My guess is still the latter because that's what I've heard the rumors about, but this article is pretty unclear on this fact.

Falcon 40B was released as a "free with royalties above a certain amount of profit" license, and got roasted for it. It was so bad that they changed the license to Apache. I don't think any business would run such a "licensed" model over MPT 30B or Falcon 40B, unless its way better than LLaMA 65b.

I think it is supposed to be better than LLaMA 65B. Plenty of businesses are paying for OAI API access.
Post reply on HN