Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

871–880 of 936 posts

Re: Open source AI is the path forward

#871

Post by BaculumMeumEst »

lostmsu wrote: post #833
BaculumMeumEst wrote: post #799

Earlier quoted context omitted.

Really? You can't even fine-tune a quantized 8B model on such a machine? That's a bummer.

You can't really fine-tune quantized models as-is. Gradients require floating-point calculations at descent precision.

I see... I'm a little out of my depth, but I did see that Meta has some documentation that talks about fine-tuning using LoRA and QLoRA, and at a glance it sounds like its doable on consumer hardware. Are you familiar with those? Am I misunderstanding? https://llama.meta.com/docs/how-to-guides/fine-tuning/

Re: Open source AI is the path forward

#872

Post by eightysixfour »

zwnow wrote: post #810
eightysixfour wrote: post #798

Earlier quoted context omitted.

I am one of those people unironically excited - the social parts of the internet have been dead and filled with bots for a long time. Now people just see it more. Maybe they’ll go outside.

Bots were easy to detect, now they're almost indistinguishable from human interaction. The death of the internet would be an incredible loss for humanity. You will not be able to trust anything you find online. Nothing is safe from this.

I am using the broad definition of bots to include large numbers of accounts controlled by small groups of people to influence online discourse.

Between those bots (for nefarious, mundane, or marketing reasons) and previous attempts at automated bots, “broad” internet discourse was already ruined. Now people recognize it. This will have the effect of pushing communities back to smaller sizes, I think this is a good thing.

People shouldn’t have trusted all the things they read online from untrusted sources in the first place.

Re: Open source AI is the path forward

#873

Post by pphysch »

tshaddox wrote: post #499
pphysch wrote: post #488

Earlier quoted context omitted.

That would be more analogous to the build toolchain than the source code, but yes

Surely traditional “open source” also needs some notion of a reproducible build toolchain, otherwise the source code itself is approximately useless. Imagine if the source code was in a programming language of which the basic syntax and semantics were known to no one but the original developers. Or more realistically, I think it’s a major problem if an open source project can only be built by an esoteric process that…

Source code in a vacuum is still valuable as a way to deal with missing/inaccurate documentation and diagnose faults and their causes.

Raw training datasets similarly has some value as you can analyze it for different characteristics to understand why the trained model is under/over-representing different concepts.

But yes real FOSS should be "open-build" and allow anyone to build a test-passing artifact from raw source material.

Re: Open source AI is the path forward

#874

Post by kristopolous »

aranke wrote: post #777
kristopolous wrote: post #773

Earlier quoted context omitted.

They're trajectory is so close to AOL's it's almost implausible. Their cash cow flagship product is widely panned by tech insiders as abusive, manipulative, and toxic but they also place significant financial resources in high quality open source projects out of what can only be described as benevolence and some commitment to the common good.

Which open-source projects is AOL known for? A quick Google search isn’t returning much.

Dropping cash for Netscape/Mozilla is the big one.

Re: Open source AI is the path forward

#876

Post by lostmsu »

BaculumMeumEst wrote: post #871
lostmsu wrote: post #833

Earlier quoted context omitted.

You can't really fine-tune quantized models as-is. Gradients require floating-point calculations at descent precision.

I see... I'm a little out of my depth, but I did see that Meta has some documentation that talks about fine-tuning using LoRA and QLoRA, and at a glance it sounds like its doable on consumer hardware. Are you familiar with those? Am I misunderstanding? https://llama.meta.com/docs/how-to-guides/fine-tuning/

Both still compute FP16 gradients. It comes back to the original point that Mac is just going to be too slow even on 8B model.

Re: Open source AI is the path forward

#877

Post by napierzaza »

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

[dead]

Re: Open source AI is the path forward

#879

Post by jlokier »

Aeolun wrote: post #685
stavros wrote: post #425

Earlier quoted context omitted.

"Open weights" means you can use the weights for free (as in beer). "Open source" means you get the training dataset and the methodology. ~Nobody does open source LLMs.

I suspect that even if you allowed people to take the data, nobody but a FAANG like organisation could even store it?

My impression is the training data for foundation models isn't that large. It won't fit on your laptop drive, but it will fit comfortably in a few racks of high-density SSDs.

Re: Open source AI is the path forward

#880

Post by jijji »

Aeolun wrote: post #685
stavros wrote: post #425

Earlier quoted context omitted.

"Open weights" means you can use the weights for free (as in beer). "Open source" means you get the training dataset and the methodology. ~Nobody does open source LLMs.

I suspect that even if you allowed people to take the data, nobody but a FAANG like organisation could even store it?

yeah, according to the article [0] about the release of Llama 3.1 405B, it was trained on 15 trillion tokens using 16000 Nvidia H100's to do it. Even if they did release the training data, I don't think many people would have the number of gpus required to actually do any real training to create the model....

[0] https://ai.meta.com/blog/meta-llama-3-1/

Post reply on HN