Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

481–490 of 965 posts

Re: Meta Llama 3

#481

Post by nmklnlknklnlk »

paxys wrote: post #240
eigenvalue wrote: post #231

I just want to express how grateful I am that Zuck and Yann and the rest of the Meta team have adopted an open approach and are sharing the model weights, the tokenizer, information about the training data, etc. They, more than anyone else, are responsible for the explosion of open research and improvement that has happened with things like llama.cpp that now allow you to run quite decent models locally on consumer h…

You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.

NVidia, AMD, Microsoft?

Re: Meta Llama 3

#482

Post by bigstrat2003 »

bearjaws wrote: post #423
lordswork wrote: post #132

Earlier quoted context omitted.

Lex walked so that Dwarkesh could run. He runs the best AI podcast around right now, by a long shot.

I feel like Lex has gone full 'both sides' at this point, waiting for him to have Alex Jones on at this point. There is no real commentary to pull from his interviews, at best you get some interesting stories but not the truth.

That is a strength, not a weakness. It's valuable to see why people, even those with whom we disagree, think the way they do. There's already far too much of a tendency to expel heretics in today's society, so the fact that Lex just patiently listens to people is a breath of fresh air.

Re: Meta Llama 3

#483

Post by bigstrat2003 »

mrtranscendence wrote: post #372
bigstrat2003 wrote: post #362

Earlier quoted context omitted.

Maybe you should consider that others may not share your views on Lex's lack of charisma or interesting thoughts.

I'll agree that "interesting thoughts" may be up to interpretation, but imma fight you on the charisma thing. I looked up "flat affect" in the dictionary and there were no words, only a full-page headshot of Lex Fridman.

I'm simply pointing out the answer to your "I don't understand why people like him" question. If you can't understand why people don't share your hatred for something, then odds are that the disconnect is because they don't share your reasons for hating it.

Re: Meta Llama 3

#484

Post by yogorenapan »

seveibar wrote: post #401

Just a quick observation: it seems to not mention commercial companies (or at least be biased against it). I tried executing "what are popular design tools with an infinite canvas" against both meta.ai and OpenAI. OpenAI returned what you would expect, Figma Sketch etc. But MetaAI only returned free/open-source software https://x.com/seveibar/status/1781042926430437404

I actually like that. I know they aren’t the “best” responses but as defaults, I would be more suspicious if it gave paid tools. I’m tested it and you can just ask for commercial tools if you want

Re: Meta Llama 3

#485

Post by saratogacx »

FrustratedMonky wrote: post #359
noiseinvacuum wrote: post #328

Earlier quoted context omitted.

He went into the details of how he thinks about open sourcing weights for Llama responding to a question from an analyst in one of the earnings call last year after Llama release. I had made a post on Reddit with some details. https://www.reddit.com/r/MachineLearning/s/GK57eB2qiz Some noteworthy quotes that signal the thought process at Meta FAIR and more broadly * We’re just playing a different game on the infrastru…

"different game" But what game? What is the AI play that makes giving it away a win for meta?

A lot of the other companies are selling AI as a service. Meta hasn't really been in the space of selling a raw service in that way. However, they are at a center point of human interaction that few can match. In this space, it is how they can leverage those models to enhance that and make that experience better that can be where they win. (Think of, for example, giving a summery of what you've missed in your groups, letting you join more and still know what's happening without needing to shift through it all, identifying events and activities happening that you'd be interested in. This will make it easier to join more groups as the cost of being in one is less, driving more engagement).

For facebook, it isn't the technology, but how it is applied, is where their game starts to get interesting.

When you give away the tooling and treat it as first class, you'll get the wider community improving it on top of your own efforts, cycle that back into the application of it internally and you now have a positive feedback loop where other, less open models, lack one.

Re: Meta Llama 3

#486

Post by littlestymaar »

jiggawatts wrote: post #475
littlestymaar wrote: post #247

Earlier quoted context omitted.

Mistral is not “open source” either since we cannot reproduce it (the training data is not published). Both are open weight models, and they are both released under a license whose legal basis is unclear: it's not actually clear if they own any intellectual property over the model at all. Of course they claim such IP, but no court has ruled on this yet AFAIK and legislators could also enact laws that make these publi…

Is “reproducibility” actually the right term here? It’s a bit like arguing that Linux is not open source because you don’t have every email Linus and the maintainers ever received. Or that you don’t know what lectures Linus attended or what books he’s read. The weights “are the thing” in the same sense that the “code is the thing”. You can modify open code and recompile it. You can similarly modify weights with fine…

> It’s a bit like arguing that Linux is not open source because you don’t have every email Linus and the maintainers ever received. Or that you don’t know what lectures Linus attended or what books he’s read.

Linux is open source, because you can actually compile it yourself! You don't need Linus's email for that (and if you needed some secret cryptographic key on Linus' laptop to decrypt and compile the kernel, then it wouldn't make sense to call it open-source either).

A language model isn't a piece of code, it's a huge binary blob that's being executed by a small piece of code that contains little of the added value, everything that matters is in the blob. Sharing only the compiled blob and the code to run makes it unsuitable for an “open source qualifier” (It's kind of the same thing as proprietary Java code: the VM is open-source but the bytecode you run on it isn't).

And yes, you can fine-tune and change things in the model weights themselves the same way you can edit the binary of a proprietary game to disable DRMs, that doesn't make it open-source either. Fine tuning doesn't give you the same level of control over the behavior of the model as the initial training does, like binary hacking doesn't give you the same control as having the source code to edit and rebuild.

Re: Meta Llama 3

#487

Post by nmklnlknklnlk »

a_wild_dandan wrote: post #425
paxys wrote: post #240

Earlier quoted context omitted.

You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.

Also, being open source adds phenomenal value for Meta: 1. It attracts the world's best academic talent, who deeply want their work shared. AI experts can join any company, so ones which commit to open AI have a huge advantage. 2. Having armies of SWEs contributing millions of free labor hours to test/fix/improve/expand your stuff is incredible. 3. The industry standardizes around their tech, driving down costs and d…

OpenAI engineers don't work for free. Facebook subsidizes their engineers because they have $20B. OpenAI doesn't have that luxury.

Re: Meta Llama 3

#488

Post by redbell »

paxys wrote: post #240
eigenvalue wrote: post #231

I just want to express how grateful I am that Zuck and Yann and the rest of the Meta team have adopted an open approach and are sharing the model weights, the tokenizer, information about the training data, etc. They, more than anyone else, are responsible for the explosion of open research and improvement that has happened with things like llama.cpp that now allow you to run quite decent models locally on consumer h…

You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.

Someone, somewhere on YT [1], coined the term Vanilla CEOs to describe non-tech-savvy CEOs, typically MBA graduates, who may struggle to innovate consistently. Unlike their tech-savvy counterparts, these CEOs tend to maintain the status quo rather than pursue bold visions for their companies..

1. https://youtu.be/gD3RV8nMzh8

Re: Meta Llama 3

#489

Post by tymscar »

raytopia wrote: post #202

I can't get it to write a pbr shader (not that I've gotten any of these models to do it correctly) but cool nonetheless.

I cant get it to write me a simple nix darwin activation script, which is something a newbie can do in a couple of minutes. No model can either, tried Claude 3 Opus, Gpt4 all variants, chat and api, llama 3, mistral/mixtral…

Re: Meta Llama 3

#490

Post by ghaff »

swatcoder wrote: post #472
maxsilver wrote: post #455

Earlier quoted context omitted.

> but driving software engineer salaries out of reach of otherwise profitable, sustainable businesses is not a good thing. I'm not convinced he's actually done that. Pretty much any 'profitable, sustainable business' can afford software developers. Software developers are paid pretty decently, but (grabbing a couple of lists off of Google) it looks like there's 18 careers more lucrative than it (from a wage perspecti…

Few viable technology businesses and non-technology busiesses with internal software departments were prepared to see their software engineers suddenly suddenly expect doctor or lawyer pay and can't effectively accomodate the change. They were largely left to rely on loyalty and other kinds of fragile non-monetary factors to preserve their existing talent and institutuonal knowledge and otherwise scavenge for scraps…

In general, it's probably not sustainable (with some exceptions like academia that have never paid that well leaving aside the top echelon and that had its own benefits) to expect that engineering generally lags behind SV software engineering. Especially with some level of remote persisting, presumably salaries/benefits equilibrate to at least some degree.
Post reply on HN