Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.1

llama.meta.com

Post reply on HN
Print view View original

51–60 of 279 posts

Re: Llama 3.1

#51

Post by ninjin »

meetpateltech wrote: post #14

Open Source AI Is the Path Forward - Mark Zuckerberg https://about.fb.com/news/2024/07/open-source-ai-is-the-path...

So are they actually making the models open now or are they staying the course with "kind of open" as they have done for LLaMA 1, 2, and 3 [1]?

[1]: https://opensource.org/blog/metas-llama-2-license-is-not-ope...

As I have stated time and again, it is perfectly fine for them to slap on whatever license they see fit as it is their work. But it would be nice if they used appropriate terms so as not to disrupt the discourse further than they have already done. I have written several walls of text why I as a researcher find Facebook's behaviour problematic so I will fall back on an old link [2] this time rather than writing it all over again.

[2]: https://news.ycombinator.com/item?id=38427832

Re: Llama 3.1

#53

Post by petercooper »

lolinder wrote: post #44
netsec_burn wrote: post #41

Today appears to be the day you can run an LLM that is competitive with GPT-4o at home with the right hardware. Incredible for progress and advancement of the technology. Statement from Mark: https://about.fb.com/news/2024/07/open-source-ai-is-the-path...

> at home with the right hardware Where the right hardware is 10x4090s even at 4 bits quantization. I'm hoping we'll see these models get smaller, but the GPT-4-competitive one isn't really accessible for home use yet. Still amazing that it's available at all, of course!

It's hardly cheap starting at about $10k of hardware, but another potential option appears to be using Exo to spread the model across a few MBPs or Mac Studios: https://x.com/exolabs_/status/1814913116704288870

Re: Llama 3.1

#54

Post by sva_ »

jcmp wrote: post #36

"Meta AI isn't available yet in your country" Hi from europe :/

You can load the page using a VPN and then turn off the VPN and the page will still work.

Re: Llama 3.1

#56

Post by sunaookami »

sva_ wrote: post #54
jcmp wrote: post #36

"Meta AI isn't available yet in your country" Hi from europe :/

You can load the page using a VPN and then turn off the VPN and the page will still work.

You can't sign in though, that worked before. Seems like they also check from which country your Facbook/Instagram account is. You can't create images without an account sadly.

Re: Llama 3.1

#57

Post by knicholes »

AaronFriel wrote: post #13
foxhop wrote: post #3

Earlier quoted context omitted.

Your going to need a lot more than a few, 800G VRAM needed

If previous quantization results hold up, fp8 will have nearly identical performance while using 405GiB for weights, but the KV cache size will still be significant. Too bad, too, I don't think my PC will fit 20 4090s (480GiB).

I've got a motherboard that will support 8!

Re: Llama 3.1

#58

Post by lelag »

The 405b model is actually competitive against closed source frontier models.

Quick comparison with GPT-4o:

    +----------------+-------+-------+
    |     Metric     | GPT-4o| Llama |
    |                |       | 3.1   |
    |                |       | 405B  |
    +----------------+-------+-------+
    | MMLU           |  88.7 |  88.6 |
    | GPQA           |  53.6 |  51.1 |
    | MATH           |  76.6 |  73.8 |
    | HumanEval      |  90.2 |  89.0 |
    | MGSM           |  90.5 |  91.6 |
    +----------------+-------+-------+

Re: Llama 3.1

#59

Post by cchance »

lelag wrote: post #58

The 405b model is actually competitive against closed source frontier models. Quick comparison with GPT-4o: +----------------+-------+-------+ | Metric | GPT-4o| Llama | | | | 3.1 | | | | 405B | +----------------+-------+-------+ | MMLU | 88.7 | 88.6 | | GPQA | 53.6 | 51.1 | | MATH | 76.6 | 73.8 | | HumanEval | 90.2 | 89.0 | | MGSM | 90.5 | 91.6 | +----------------+-------+-------+

Super cool, though sadly 405b will be outside most personal usage without cloud providers which sorta defeats the purpose of opensource to some extent atleast sadly, because .. nvidia's rampup of consumer VRAM is glacial

Re: Llama 3.1

#60

Post by monkmartinez »

jcmp wrote: post #36

"Meta AI isn't available yet in your country" Hi from europe :/

Why are (some) Europeans surprised when they are not included in tech product débuts? My lay understanding could best be described as; EU law is incredibly business unfriendly and takes a heroic effort in time and money to implement the myriad of requirements therein. Am I wrong?
Post reply on HN