Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

701–710 of 965 posts

Re: Meta Llama 3

#701

Post by fragmede »

bevekspldnw wrote: post #633
observationist wrote: post #67

https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…

I don’t understand how the idea of open source become some sort of pseudo-legalistic purity test on everything. Models aren’t code, some of the concepts of open source code don’t map 1:1 to freely available models. In spirit I think this is “open source”, and I think that’s how the majority of people think. Turning everything into some sort of theological debate takes away a lot of credit that Meta deserves. Google i…

> In spirit I think this is “open source”, and I think that’s how the majority of people think.

No, it isn't. You do, but, as evidenced by other comments, there's clearly people that don't. Thinking that you're with the majority and it's just a vocal minority is one thing, but it could just as easily be said that the vocal groups objecting to your characterization are representative of the mainstream view.

If we look at these models as the output of a compiler, that we don't have the inputs to, but that we are free (ish) to use and modify and redistribute, it's a nice grant from the copyright holder, but that very much doesn't look like open source. Open source, applied to AI models would mean giving us (a reference to) the dataset and the code used to train the model so we could tweak it to train the model slightly differently. To be less apologetic or something by default, instead of having to give it additional system instructions.

Model Available(MA) is freer than Model unavailable, and it's more generous than model unavailable, but it's very much not in the spirit of open source. I can't train my own model using what Meta has given us here.

And just to note, Google Gemma is the one they are releasing weights for. They are doing this and deserve credit for it.

Re: Meta Llama 3

#702

Post by [deleted] »

aussieguy1234 wrote: post #554

"You’ll also soon be able to test multimodal Meta AI on our Ray-Ban Meta smart glasses." Now this is interesting. I've been thinking for some time now that traditional computer/smartphone interfaces are on the way out for all but a few niche applications. Instead, everyone will have their own AI assistant, which you'll interact with naturally the same way as you interact with other people. Need something visual? Just…

[deleted]

Re: Meta Llama 3

#703

Post by dlandau »

asadotzler wrote: post #675
agar wrote: post #513

Earlier quoted context omitted.

If you actually listen to how Zuck defines the metaverse, it's not Horizons or even a VR headset. That's what pundits say, most of whom love pointing out big failures more than they like thinking deeply. He sees the metaverse as the entire shared online space that evolves into a more multi-user collaborative model with more human-centric input/output devices than a computer and phone. It includes co-presence, mixed r…

50 billion dollars and fewer than 10 million MAU. That's a massive failure.

A chunky portion of those dollars were spent on buying and pre-ordering GPUs that were used to train and serve LLaMa

Re: Meta Llama 3

#704

Post by pama »

littlestymaar wrote: post #494
touisteur wrote: post #477

Earlier quoted context omitted.

I have a hard time about the "cannot reproduce" categorization. There are places (e.g. in the Linux kernel? AMD drivers?) where lots of generated code is pushed and (apart from the rants of huge unwieldy commits and complaints that it would be better engineering-wise to get their hands on the code generator, it seems no one is saying the AMD drivers aren't GPL compliant or OSI-compliant? There are probably lots of OS…

But with generated code what you end up with is still code, that can be edited by whoever needs. If AMD stopped maintaining their drivers then people would be maintaining the generated code, it wouldn't be a nice situation but it would work, whereas model weights are akin to the binary blobs you get in the Android world, binary blobs that nobody call open-source…

I personally think that the model artifacts are simply programs with tons of constants. Many math routines have constants in their approximations and I don’t expect the source to include the full derivation for these constants all the time. I see LLMs as a same category but with (much) larger sets of parameters. What is better about the LLMs than some of the mathematical constants in complicated function approximations, is that I can go and keep training an LLM whereas the math/engineering libraries might not make it easy for me to modify them without also figuring out the details that led to those particular parameter choices.

Re: Meta Llama 3

#705

Post by KingOfCoders »

observationist wrote: post #67

https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…

"Llama is not open source."

This is interesting. Can you point me to an OSI discussion what would constitute an open source license for LLMs? Obviously they have "source" (network definitions) and "training data" and "weights".

I'm not aware of any such discussion.

Re: Meta Llama 3

#706

Post by aerialfish »

tmalsburg2 wrote: post #352
filoleg wrote: post #336

Earlier quoted context omitted.

Having an engineering mindset is not the same as never making mistakes (or never being too early to the market). The only way you won’t make those mistakes and keep a perfect record is if you never do anything major or step out of the comfort zone. If Apple didn’t try and fail with Newton[0] (which was too early to the market for many reasons, both tech-related and not), we might’ve not had iPhone today. The engineer…

His engineering mindset made him blind to the fact the metaverse was a product that nobody wanted or needed. In one of the Fridman interviews, he goes on and on about all the cool technical challenges involved in making the metaverse work. But when Fridman asked him what he likes to do in his spare time, it was all things that you could precisely not do in the metaverse. It was baffling to me that he failed to connec…

Yes, I thought the same exact thing. Seemed so odd to hear him gush over his foiling and MMA while simultaneously expecting everyone else to migrate to the metaverse.

Re: Meta Llama 3

#707

Post by mgraczyk »

whereistimbo wrote: post #692

How do they plan to make money with this? They can even make money with their 24K GPU cluster as IaaS if they want to. Even Google is gatekeeping its best Gemini model behind. https://web.archive.org/web/20240000000000*/https://filebin.... https://web.archive.org/web/20240419035112/https://s3.filebi...

Facebook does not lease hardware like that because (what I was told during bootcamp) "the best return on Capital we can get from our hardware is adding more compute to facebook.com"

Re: Meta Llama 3

#708

Post by agleason »

whereistimbo wrote: post #692

How do they plan to make money with this? They can even make money with their 24K GPU cluster as IaaS if they want to. Even Google is gatekeeping its best Gemini model behind. https://web.archive.org/web/20240000000000*/https://filebin.... https://web.archive.org/web/20240419035112/https://s3.filebi...

Meta makes money by selling ads. they want people to be more glued into their platforms and sharing stuff. they hope that people will use their model to make content to share

Re: Meta Llama 3

#709

Post by fragmede »

bevekspldnw wrote: post #700
skybrian wrote: post #644

Earlier quoted context omitted.

It doesn’t mean it’s a bad license, just that it doesn’t meet the definition. There are legitimate reasons for companies to use source-available licenses. You still get to see the source code and do some useful things with it, but read the terms to see what you can do. Meanwhile, there are also good reasons not to water down a well-defined term so it becomes meaningless like “agile” or “open.” This gets confusing bec…

But it’s also a bit absurd in a sense - let’s say you have all of Meta’s code and training data. Ok, now what? Even if you also had a couple spare data centers, unlimited money, and an army of engineers, you can’t even find enough NVIDIA cards to do the training run. This isn’t some homebrew shit, it’s millions upon millions of dollars of computational power devoted to building this thing. I think at a fundamental le…

People are thinking what open really means, and they're telling you this isn't open. it definitely isn't Open Source, as defined by the OSI.

Open Source has a specific meaning and this doesn't meet it. It's generous of Meta to give us these models and grant us access to them, and let us modify them, fine tune them, and further redistribute them. It's really great! But we're still in the dark as to how they came about the weights. It's a closed, proprietary process, of which we have some details, which is interesting and all, but that's not the same as having access to the actual mechanism used to generate the model.

Re: Meta Llama 3

#710

Post by qwertox »

whereistimbo wrote: post #692

How do they plan to make money with this? They can even make money with their 24K GPU cluster as IaaS if they want to. Even Google is gatekeeping its best Gemini model behind. https://web.archive.org/web/20240000000000*/https://filebin.... https://web.archive.org/web/20240419035112/https://s3.filebi...

I am paying for ChatGPT. And I'm very willing to switch away from it for the same price because it is so unreliable, as in network problems, very sluggish performance.

But currently none matches its quality and data export capabilities.

Post reply on HN