Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.1

llama.meta.com

Post reply on HN
Print view View original

231–240 of 279 posts

Re: Llama 3.1

#231

Post by TeMPOraL »

drexlspivey wrote: post #223
nomel wrote: post #166

Earlier quoted context omitted.

Is there any other case where "open source" is used for something that can't be reproduced? Seems like a new term is required, in the concept of "open source, non-reproducible artifacts". I suppose language changes. I just prefer it changes towards being more precise, not less.

No, the term is fine, “source” in “open source” refers to source code. A dataset by definition is not source code. Stop changing the meaning of words.

A dataset very much is the source code. It's the part that gets turned into the program through an automated process (training is equivalent to compilation).

Re: Llama 3.1

#232

Post by ninjin »

j_maffe wrote: post #164
ninjin wrote: post #155

Earlier quoted context omitted.

Good, then we can expect them to call it what it is then? Not open source and not open science and a regression in terms of openness in relationship to what came before. Because that is precisely my objection. There are those of us that have been committed to those ideals for a long time and now one of the largest corporations on earth is appropriating those terms for marketing purposes.

I think it's great that you're fighting to maintain the term's fundamental meaning. I do, however, think that we need to give credit where credit is due to companies who take actions in the right direction to encourage more companies to do the same. If we blindly protest any positive-impact action by corporations for not being perfect, they'll get the hint and stop trying to appease the community entirely.

I am in agreement. However, I do believe that a large portion of the community here is also missing a key point: Facebook was more open five years ago with their AI research than they are today. I suspect this perspective is because of the massive influx of people into AI around the time of the ChatGPT release. From their point of view, Facebook's move (although dishonestly labelled as something it is not) is a step in the right direction relative to "Open"AI and others. While for us that have been around for longer, openness "peaked" around 2018 and has been in steady decline ever since. If you see the wall of text I linked in my first comment in this chain, there is a longer description of this historical perspective.

It should also be noted (again) that the value of the terms open science and open source comes from the sacrifices and efforts of numerous academic, commercial, personal, etc. actors over several decades. They "paid" by sticking to the principles of these movements and Facebook is now cashing in on their efforts; solely for their own benefit. Not even Microsoft back in 2001 in the age of "fear uncertainty and doubt" were so dishonest as to label the source-available portions of their Shared Source Initiative as something it was not. Facebook has been called out again and again since the release of LLaMA 1 (which in its paper appropriated the term "open") and have shown no willingness to reconsider their open science and open source misuse. At this point, I can no longer give them the benefit of the doubt. The best defence I have heard is that they seek to "define open in the 'age of AI'", but if that was the case, where is their consensus building efforts akin to what we have seen numerous academics and OSI carry out? No, sadly the only logical conclusion is that it is cynical marketing on their part, both from their academics and business people.

[1]: https://en.wikipedia.org/wiki/Shared_Source_Initiative

In short. I think the correct response to Facebook is: "Thank you for the weights, we appreciate it. However, please stop calling your actions and releases something they clearly are not."

Re: Llama 3.1

#233

Post by TeMPOraL »

xu_ituairo wrote: post #174
nomel wrote: post #166

Earlier quoted context omitted.

Is there any other case where "open source" is used for something that can't be reproduced? Seems like a new term is required, in the concept of "open source, non-reproducible artifacts". I suppose language changes. I just prefer it changes towards being more precise, not less.

This feels somewhat analogous to games like Quake being open-sourced though still needing the user to provide the original game data files.

But games like Quake are not "open source". They have been open-sourced, specifically the executable parts were, without the assets. This is usually spelled out clearly as the process happen.

In terms of functional role, if we're to compare the models to open-sourced games, then all that's been open-sourced is the trivial[0] bit of code that does the inference.

Maybe a more adequate comparison would be a SoC running a Linux kernel with a big NVidia or Qualcomm binary blob in the middle of it? Sure, the Linux kernel is open source, but we wouldn't call the SoC "open source", because all that makes it what it is (software-side) is hidden in a proprietary binary.

--

[0] - In the sense that there's not much of it, and it's possible to reproduce from papers.

Re: Llama 3.1

#234

Post by TeMPOraL »

llm_trw wrote: post #210
nomel wrote: post #154

Earlier quoted context omitted.

I really wish people would use "open weights" rather than "open source". It's precise and obvious, and leaves an accurate descriptor for actual "open source" models, where the source and methods that that generate the artifact , that is the weights, is open.

> where the source and methods that that generate the artifact, that is the weights, is open. When you require the same thing in software, namely the whole stack to run the software in question to be open source, we don't call the license open source.

Nope. Those model releases only open source the equivalent of "run.bat" that does some trivial things and calls into a binary blob. We wouldn't call such a program "open source".

Hell, in case of the models, "the whole stack to run the software" already is open source. Literally everything except the actual sources - the datasets and the build scripts (code doing the training) - is available openly. This is almost a literal inverse of "open source", thus shouldn't be called "open source".

Re: Llama 3.1

#235

Post by [deleted] »

mi_lk wrote: post #191
lelag wrote: post #58

The 405b model is actually competitive against closed source frontier models. Quick comparison with GPT-4o: +----------------+-------+-------+ | Metric | GPT-4o| Llama | | | | 3.1 | | | | 405B | +----------------+-------+-------+ | MMLU | 88.7 | 88.6 | | GPQA | 53.6 | 51.1 | | MATH | 76.6 | 73.8 | | HumanEval | 90.2 | 89.0 | | MGSM | 90.5 | 91.6 | +----------------+-------+-------+

How do you draw/generate such ascii table?

[deleted]

Re: Llama 3.1

#236

Post by diffeomorphism »

api wrote: post #146
paxys wrote: post #137

Earlier quoted context omitted.

You don't need a model of this scale for personal use. Llama 3.1 8B can easily run on your laptop right now. The 70B model can run on a pair of 4090s.

I have the 70b model running quantized just fine on an M1 Max laptop with 64GiB unified RAM. Performance is fine and so far some Q&A tests are impressive. This is good enough for a lot of use cases... on a laptop. An expensive laptop, but hardware only gets better and cheaper over time.

Just for reference, the current version of that laptop costs 4800€ (14 inch macbook pro, m3 max, 64gb of ram, 1TB of storage). So price-wise that is more like four laptops.

Re: Llama 3.1

#237

Post by OrangeMusic »

TeMPOraL wrote: post #227
fngjdflmdflg wrote: post #160

Earlier quoted context omitted.

As far as I know it's not just the weights. it's everything but the dataset. So the code used to generate the weights is also open source.

In other words, it's everything except the one thing that actually matters .

Maybe, but it doesn't mean it's not open source.

Re: Llama 3.1

#238

Post by yard2010 »

lawlessone wrote: post #10
TechDebtDevin wrote: post #2

Nice, someone donate me a few 4090s :(

maybe someone will figure out some ways to prune/ quantize it a huge amount ;-; edit: If the AI bubble pops we will be swimming in GPUs... but no new models.

This bubble collapsing along with most blockchains going all in with proof of stake rather than proof of work is myself and every other gamer wet dream.

Re: Llama 3.1

#239

Post by v3ss0n »

ofou wrote: post #215

Llama 3 Training System 19.2 exaFLOPS _____ / \ Cluster 1 Cluster 2 / \ 9.6 exaFLOPS 9.6 exaFLOPS / \ _______ _______ / ___ \ / \ / \ ,----' / \`. `-' 24000 `--' 24000 `----. ( _/ __) GPUs GPUs ) `---'( / ) 400+ TFLOPS 400+ TFLOPS ,' \ ( / per GPU per GPU ,' \ \/ ,' \ \ TOTAL SYSTEM ,' \ \ 19,200,000 TFLOPS ,' \ \ 19.2 exaFLOPS ,' \___\ ,' `----------------'

how much would it cost?

Re: Llama 3.1

#240

Post by Alifatisk »

listic wrote: post #199
foundval wrote: post #74

You can chat with these new models at ultra-low latency at groq.com. 8B and 70B API access is available at console.groq.com. 405B API access for select customers only – GA and 3rd party speed benchmarks soon. If you want to learn more, there is a writeup at https://wow.groq.com/now-available-on-groq-the-largest-and-m... . (disclaimer, I am a Groq employee)

Just checked it out. Is pay-as-you-go API access available at all? It says 'Coming Soon' https://console.groq.com/settings/billing

I think you answered it yourself? It’s coming soon, so it is not available now, but soon.
Post reply on HN