Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.1

llama.meta.com

Post reply on HN
Print view View original

61–70 of 279 posts

Re: Llama 3.1

#61

Post by lolinder »

jcmp wrote: post #36

"Meta AI isn't available yet in your country" Hi from europe :/

Competition is a funny thing—it doesn't just apply to companies competing for customers, it also applies to governments competing for companies to make products available to their citizens. Turns out that if you make compliance with your laws onerous enough they can actually just choose to opt out of your country altogether, or at a minimum delay release in your country until they can check all your boxes.

The only solution is a worldwide government that can impose laws in all countries at once, but that's unlikely to happen any time soon.

Re: Llama 3.1

#62

Post by [deleted] »

netsec_burn wrote: post #41

Today appears to be the day you can run an LLM that is competitive with GPT-4o at home with the right hardware. Incredible for progress and advancement of the technology. Statement from Mark: https://about.fb.com/news/2024/07/open-source-ai-is-the-path...

[deleted]

Re: Llama 3.1

#63

Post by pat2man »

lolinder wrote: post #8
foxhop wrote: post #3

Earlier quoted context omitted.

Your going to need a lot more than a few, 800G VRAM needed

Quantized to 4 bits you'll only need ~200GB! 5 4090s should cover it.

Two 128gb Mac studios networked via thunderbolt 4?

Re: Llama 3.1

#66

Post by Zambyte »

ninjin wrote: post #51
meetpateltech wrote: post #14

Open Source AI Is the Path Forward - Mark Zuckerberg https://about.fb.com/news/2024/07/open-source-ai-is-the-path...

So are they actually making the models open now or are they staying the course with "kind of open" as they have done for LLaMA 1, 2, and 3 [1]? [1]: https://opensource.org/blog/metas-llama-2-license-is-not-ope... As I have stated time and again, it is perfectly fine for them to slap on whatever license they see fit as it is their work. But it would be nice if they used appropriate terms so as not to disrupt the disco…

> it is perfectly fine for them to slap on whatever license they see fit as it is their work.

Is it? Has there been a ruling on the enforceability of the license they attach to their models yet? Just because you say what you release can only be used for certain things doesn't actually mean what you say means anything.

Re: Llama 3.1

#67

Post by monkmartinez »

daft_pink wrote: post #55

What kind of machine do I need to run 405B local?

You can't. Sorry.

Unless...

You have a couple hundred $k sitting around collecting dust... then all you need is a DGX or HGX level of vRAM, the power to run it, the power to keep it cool, and place for it to sit.

Re: Llama 3.1

#68

Post by kingsleyopara »

cchance wrote: post #59
lelag wrote: post #58

The 405b model is actually competitive against closed source frontier models. Quick comparison with GPT-4o: +----------------+-------+-------+ | Metric | GPT-4o| Llama | | | | 3.1 | | | | 405B | +----------------+-------+-------+ | MMLU | 88.7 | 88.6 | | GPQA | 53.6 | 51.1 | | MATH | 76.6 | 73.8 | | HumanEval | 90.2 | 89.0 | | MGSM | 90.5 | 91.6 | +----------------+-------+-------+

Super cool, though sadly 405b will be outside most personal usage without cloud providers which sorta defeats the purpose of opensource to some extent atleast sadly, because .. nvidia's rampup of consumer VRAM is glacial

You might be able to get away with running a heavily quantizied 405b model using CPU inference at a blistering fast token every 5 seconds on a 7950x.

Re: Llama 3.1

#69

Post by duchenne »

cchance wrote: post #59
lelag wrote: post #58

The 405b model is actually competitive against closed source frontier models. Quick comparison with GPT-4o: +----------------+-------+-------+ | Metric | GPT-4o| Llama | | | | 3.1 | | | | 405B | +----------------+-------+-------+ | MMLU | 88.7 | 88.6 | | GPQA | 53.6 | 51.1 | | MATH | 76.6 | 73.8 | | HumanEval | 90.2 | 89.0 | | MGSM | 90.5 | 91.6 | +----------------+-------+-------+

Super cool, though sadly 405b will be outside most personal usage without cloud providers which sorta defeats the purpose of opensource to some extent atleast sadly, because .. nvidia's rampup of consumer VRAM is glacial

Most SMBs would be able to run it. This is already a huge win for decentralized AI.

Re: Llama 3.1

#70

Post by aabhay »

cchance wrote: post #59
lelag wrote: post #58

The 405b model is actually competitive against closed source frontier models. Quick comparison with GPT-4o: +----------------+-------+-------+ | Metric | GPT-4o| Llama | | | | 3.1 | | | | 405B | +----------------+-------+-------+ | MMLU | 88.7 | 88.6 | | GPQA | 53.6 | 51.1 | | MATH | 76.6 | 73.8 | | HumanEval | 90.2 | 89.0 | | MGSM | 90.5 | 91.6 | +----------------+-------+-------+

Super cool, though sadly 405b will be outside most personal usage without cloud providers which sorta defeats the purpose of opensource to some extent atleast sadly, because .. nvidia's rampup of consumer VRAM is glacial

Zoom out a bit. There’s a massive feeder ecosystem around llama. You’ll see many startups take this on and help drive down inference costs for everyone and create competitive pressure that will improve the state of the art.
Post reply on HN