Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

751–760 of 936 posts

Re: Open source AI is the path forward

#751

Post by Ukv »

dev1ycan wrote: post #541

Reality: they've realize gpt 4 is a wall, they can't keep pouring trillions of dollars into it for no improvement or little at all, so now they want to put it to the open source until someone figures out the next step then they'll take it behind closed doors again. I hate how the moment it's too late will be, by design, closed doors.

> Reality: they've realize gpt 4 is a wall, they can't keep pouring trillions of dollars into it for no improvement or little at all, so now they want to put it to the open source [...]

This is Meta (LLaMA, which has had available weights for a while), not OpenAI (GPT).

Re: Open source AI is the path forward

#752

Post by baq »

have_faith wrote: post #729
sensanaty wrote: post #718

Meanwhile Facebook is flooded with AI-generated slop with hundreds of thousands of other bots interacting with it to boost it to whoever is insane enough to still use that putrid hellhole of a mass-data-harvesting platform. Dead internet theory is very much happening in real time, and I dread what's about to come since the world has collectively decided to lose their minds with this AI crap. And people on this site a…

The feed certainly is, but I suspect most activity left on Facebook is happening in group pages. Groups are the only thing I still log in for as some of them, particularly the local ones, have no other way of taking part. They are also commonly membership by request and actively moderated. If I had the time (and energy) I might put some effort into advocating to moving to something else, but it will be an uphill batt…

I stopped going on facebook a few years ago and don't miss it; I don't even need messenger as everyone migrated to whatsapp (yes I know, normal people don't want to move to signal, but got quite a few techy friends to migrate). The FB-only groups are indeed a problem, I'm delegating them to my wife.

IF I ever had to go to FB for anything, I'd probably install a wall-removing browser extension. Mobile app is of course out of question.

Re: Open source AI is the path forward

#753

Post by alickz »

Geezus_42 wrote: post #614
hluska wrote: post #307

Earlier quoted context omitted.

Look at it from the perspective of an elected official: If it succeeds, you were ahead of the curve. If it fails, you were prudent enough to fund an investigation early. Either way, bleeding edge tech gives you a W.

Or you wasted a bunch of tax payer money on some over hyped and over funded nonsense.

how would you determine that without investigation?

Re: Open source AI is the path forward

#754

Post by kataklasm »

jart wrote: post #408
tambourine_man wrote: post #264

Earlier quoted context omitted.

Praising is good. Gratitude is a bit much. They got this big by selling user generated content and private info to the highest bidder. Often through questionable means. Also, the underdog always touts Open Source and standards, so it’s good to remain skeptical when/if tables turn.

I'm perfectly happy with them draining the life essence out of the people crazy enough to still use Facebook, if they're funneling the profits into advancing human progress with AI. It's an Alfred Nobel kind of thing to do.

It's not often you see a take this bad on HN. Wow!

You are aware Facebook tracks everyone, not just people with Facebook accounts, right? They have a history of being anti-consumer in every sense of the word. So while I can understand where you're coming from, it's just not anywhere close to being reality.

If you want to or not, if you consent or not, Facebook is tracking and selling you.

Re: Open source AI is the path forward

#755

Post by fishermanbill »

Its not open source.

We don't get the data or training code. The small runtime framework is open source but that's of little use as its largely fixed in implementation due to the weights. Yes we can fine tune but that is akin to modifying video games - we can do it but there's only so much you can do within reasonable effort and no one would call most video games 'open source'*.

Its freeware and Meta's strategy is much more akin to the strategy Microsoft used with Internet Explorer to capture the web browser market. No one was saying god bless Microsoft for trying to capture the browser market with I.E. Nothing wrong with Meta's strategy just don't call it open source.

*weights are data and so is the video/audio output of a video game. If we gave away that video game output for free we wouldn't call the video game open source as the myriad freeware games essentially do.

Re: Open source AI is the path forward

#756

Post by exe34 »

jononor wrote: post #734
exe34 wrote: post #690

Earlier quoted context omitted.

how do you train it in a different language by changing the training code?

By selecting different dataset. Of course this dataset does need to exist. In practice building and curating datasets also involves a lot of code.

sounds like you need the data to train the model.

Re: Open source AI is the path forward

#758

Post by Palmik »

fishermanbill wrote: post #755

Its not open source. We don't get the data or training code. The small runtime framework is open source but that's of little use as its largely fixed in implementation due to the weights. Yes we can fine tune but that is akin to modifying video games - we can do it but there's only so much you can do within reasonable effort and no one would call most video games 'open source'*. Its freeware and Meta's strategy is mu…

I don't think these analogies work.

Meta provides open source code to modify the the weights (fine tune the model). In this context, fine-tuning the model is better converted to being able to modify the code of the game.

Re: Open source AI is the path forward

#759

Post by fishermanbill »

Palmik wrote: post #758
fishermanbill wrote: post #755

Its not open source. We don't get the data or training code. The small runtime framework is open source but that's of little use as its largely fixed in implementation due to the weights. Yes we can fine tune but that is akin to modifying video games - we can do it but there's only so much you can do within reasonable effort and no one would call most video games 'open source'*. Its freeware and Meta's strategy is mu…

I don't think these analogies work. Meta provides open source code to modify the the weights (fine tune the model). In this context, fine-tuning the model is better converted to being able to modify the code of the game.

So do video game developers (provide source code to modify their games) the analogy absolutely works. I can list a huge amount of actually open source software that I can see the source code and data for which is very different from Llama etc.

Re: Open source AI is the path forward

#760

Post by sharpshadow »

oersted wrote: post #146
blackeyeblitzar wrote: post #77

Earlier quoted context omitted.

What about distributed training on volunteer hardware? Is that feasible?

It is an exciting concept, there's a huge wealth of gaming hardware deployed that is inactive at most hours of the day. And I'm sure people are willing to pay well above the electricity cost for it. Unfortunately, the dominant LLM architecture makes it relatively infeasible right now. - Gaming hardware has too limited VRAM for training any kind of near-state-of-the-art model. Nvidia is being annoyingly smart about th…

Couldn’t VRAM be subsidised with SSDs on a lower end machine? It would make it slower but maybe useful at last.
Post reply on HN