Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.1

llama.meta.com

Post reply on HN
Print view View original

111–120 of 279 posts

Re: Llama 3.1

#111

Post by lolinder »

Daunk wrote: post #105
monkmartinez wrote: post #60

Earlier quoted context omitted.

Why are (some) Europeans surprised when they are not included in tech product débuts? My lay understanding could best be described as; EU law is incredibly business unfriendly and takes a heroic effort in time and money to implement the myriad of requirements therein. Am I wrong?

Most things do dèbute in the EU, unless the product or company behind it doesn't value your privacy. Meta does not value your privacy.

Privacy was the first thing that the EU did that started this trend of companies slowing their EU releases because of GDPR. Now there's the Digital Markets Act and the AI Act that both have caused companies to slow their releases to the EU.

Each new large regulation adds another category of company to the list of those who choose not to participate. Sure, you can always label them as companies who don't value principle X, but at some point it stops being the fault of the companies and you have to start looking at whether there are too many enormous regulations slowing down tech releases.

Re: Llama 3.1

#112

Post by lossolo »

changoplatanero wrote: post #27
hubraumhugo wrote: post #9

I wrote about this when llama-3 came out, and this launch confirms it: Meta's goal from the start was to target OpenAI and the other proprietary model players with a "scorched earth" approach by releasing powerful open models to disrupt the competitive landscape. Meta can likely outspend any other AI lab on compute and talent: - OpenAI makes an estimated revenue of $2B and is likely unprofitable. Meta generated a rev…

> Open source likely attracts better talent and researchers I work at OpenAI and used to work at meta. Almost every person from meta that I know has asked me for a referral to OpenAI. I don’t know anyone who left OpenAI to go to meta.

When was that?

Re: Llama 3.1

#113

Post by lawlessone »

sunaookami wrote: post #56
sva_ wrote: post #54

Earlier quoted context omitted.

You can load the page using a VPN and then turn off the VPN and the page will still work.

You can't sign in though, that worked before. Seems like they also check from which country your Facbook/Instagram account is. You can't create images without an account sadly.

Someone will torrent it soon enough i'm sure.

Re: Llama 3.1

#114

Post by kkielhofner »

tintor wrote: post #93
changoplatanero wrote: post #27

Earlier quoted context omitted.

> Open source likely attracts better talent and researchers I work at OpenAI and used to work at meta. Almost every person from meta that I know has asked me for a referral to OpenAI. I don’t know anyone who left OpenAI to go to meta.

What % of them were from FAIR vs non-FAIR?

Sample size of one but I know someone who went from FAIR to OpenAI.

Re: Llama 3.1

#115

Post by thntk »

unraveller wrote: post #104

What are the substantial changes from 3.0 to 3.1 (70B) in terms of training approach? They don't seem to say how the training data differed just that both were 15T. I gather 3.0 was just a preview run and 3.1 was distilled down from the 405B somehow.

Correct me if I'm wrong, my impression is that 3.1 is a better fine-tuned variant of base 3.0 with extensive use of synthetic data.

Re: Llama 3.1

#116

Post by tommy_axle »

dado3212 wrote: post #108

> We use synthetic data generation to produce the vast majority of our SFT examples, iterating multiple times to produce higher and higher quality synthetic data across all capabilities. Additionally, we invest in multiple data processing techniques to filter this synthetic data to the highest quality. This enables us to scale the amount of fine-tuning data across capabilities. [0] Have other major models explicitly…

It's in the https://huggingface.co/microsoft/Phi-3-mini-4k-instruct

Re: Llama 3.1

#117

Post by HanClinto »

kingsleyopara wrote: post #75

The biggest win here has to be the context length increase to 128k from 8k tokens. Till now my understanding is there hasn't been any open models anywhere close to that.

It is notable, but it's not alone. Mistral NeMo just released last week with a 128k context window:

https://news.ycombinator.com/item?id=40996058

Re: Llama 3.1

#118

Post by WinstonSmith84 »

sunaookami wrote: post #56
sva_ wrote: post #54

Earlier quoted context omitted.

You can load the page using a VPN and then turn off the VPN and the page will still work.

You can't sign in though, that worked before. Seems like they also check from which country your Facbook/Instagram account is. You can't create images without an account sadly.

I changed my Facebook country (to Canada), using also a VPN to Canada, but that didn't help. That used to work before somehow.

Re: Llama 3.1

#119

Post by gaogao »

stuckinhell wrote: post #110
cchance wrote: post #59

Earlier quoted context omitted.

Super cool, though sadly 405b will be outside most personal usage without cloud providers which sorta defeats the purpose of opensource to some extent atleast sadly, because .. nvidia's rampup of consumer VRAM is glacial

100% reddit is full of people trying to solder more vram

I've been wondering if you could just attach a chunk of vram over NVLink, since that's very roughly what FSDP is doing here anyways.

Re: Llama 3.1

#120

Post by downvotetruth »

lolinder wrote: post #8
foxhop wrote: post #3

Earlier quoted context omitted.

Your going to need a lot more than a few, 800G VRAM needed

Quantized to 4 bits you'll only need ~200GB! 5 4090s should cover it.

If an implementation had NVidia's Heterogeneous Memory Management implemented, then 192 GB RAM DDR5 + GPU VRAM would seem to be close.
Post reply on HN