Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

961–965 of 965 posts

Re: Meta Llama 3

#961

Post by iamlearningai »

lawlessone wrote: post #854
ktzar wrote: post #802

Earlier quoted context omitted.

even if they released them, wouldn't it be prohibitively expensive to reproduce the weights?

1.3 million GPU hrs for the 8b model. Take you around 130 years to train on a desktop lol.

Interesting. LLAMA is trained using 16K GPUs so it would have taken around a quarter for them. An hour of GPU use costs $2-$3 so training a custom solution using LLAMA should be atleast $15K to $1M. I am trying to get started with this thing. A few guys suggested 2 GPUs were a good start but I think that would only be good for 10K training samples.

Re: Meta Llama 3

#962

Post by ChristophGeske »

namanski wrote: post #958
ChristophGeske wrote: post #623

Earlier quoted context omitted.

Thanks for the link I just tested them and they also weark in europe without the need to start a VPN. What specs are needed to run these models. I mean the llama 70B and the Wizard 8Bx22 model. On your site they run very nicely and the answears they provide are really good they booth passed my small test and I would love to run one of them locally. So far I only ran 8B models on my 16GB RAM pc using LM Studio but hav…

Hey Christoph, thanks for trying it out - we're running this on the cloud, particularly GCP, on A100s (80g). On your query about running these models locally, I'm not sure if just upgrading your RAM would have the same throughput as what you see on the website. You can upgrade your RAM but you might get pretty bad tokens/sec.

Thanks for the reply.

I am currently testing the limits and got llama 3 70B in a 2bit-quantized form to run on my laptop with very low specs RTX3080 8GB VRAM (laptop version) and 16GB system RAM. It runs with 1,2 tokens/s which is a bit slow. The biggest issue however is the time it takes for the first token to be printed which fluctuates and takes between 1.8s to 45s.

I tested the same model on a 4070 with 16GB VRAM (desktop pc version) and 32GB system RAM and it runs at about 3-4 tokens per second. The 4070 also has the issue with quite long time for the first token to be displayed i think it was around 12s in my limited testinh.

I still try to find out how to speed the time to initial token up. 4 tokens a second is usable for many cases because that's about reading speed.

There are also 1bit-quantized 70B models appearing so there might be ways to make it even a bit faster on consumer GPUs.

I think we are at the bare edge of usability here and I keep testing.

I can not tell exactly how this strong quantization affects output quality information about that is mixed and seems to depand on the form of quantization as well.

Re: Meta Llama 3

#963

Post by murderfs »

int_19h wrote: post #931
murderfs wrote: post #615

Earlier quoted context omitted.

Because the EU requires them not to: https://ec.europa.eu/information_society/newsroom/image/docu...

This says "high-risk AI system", which is defined here: https://digital-strategy.ec.europa.eu/en/policies/regulatory... . I don't see why it would be applicable.

The text of the law says that the actual criteria can change to be whatever they think is scary:

    As regards stand-alone AI systems, namely high-risk AI systems other than those that are
   safety components of products, or that are themselves products, it is appropriate to classify
   them as high-risk if, in light of their intended purpose, they pose a high risk of harm to the
   health and safety or the fundamental rights of persons, taking into account both the severity
   of the possible harm and its probability of occurrence and they are used in a number of
   specifically pre-defined areas specified in this Regulation. The identification of those
   systems is based on the same methodology and criteria envisaged also for any future
   amendments of the list of high-risk AI systems that the Commission should be
   empowered to adopt, via delegated acts, to take into account the rapid pace of
   technological development, as well as the potential changes in the use of AI systems.
And there's also a section about systemic risks, which llama definitely falls into, and which mandates that they go through basically the same process, with offices and panels that do not yet exist:

https://ec.europa.eu/commission/presscorner/detail/en/qanda_....

Re: Meta Llama 3

#964

Post by agar »

simonklitj wrote: post #744
agar wrote: post #496

Earlier quoted context omitted.

Nor could I. And I can't imagine sitting next to my wife watching a football game together on my phone. But I could while waiting in line by myself. Similarly, I could imagine sitting next to my daughter, who is 2,500 miles away at college, watching the name together on a virtual screen we both share. And then playing mini-golf or table tennis together. Different tools are appropriate for different use cases. Don't d…

Yes, these are all very good points. You’ve got me awaiting the future of the tech a bit more eagerly.

FYI, those use cases are the present, not the future, of tech.

Co-watching TV? Big Screen: https://www.bigscreenvr.com/software

Mini-Golf? Walkabout Mini Golf: https://www.mightycoconut.com/minigolf

Table Tennis? Eleven Table Tennis: https://elevenvr.com/en/

All are amazing, polished experiences in VR that give you a sense of being "present" with someone a continent away.

Re: Meta Llama 3

#965

Post by aurbxyajwur »

freehorse wrote: post #389
observationist wrote: post #67

https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…

What are the practical use cases where the license prohibits people from using llama models? There are plenty of startups and companies that already build their business on llamas (eg phind.com). I do not see the issues that you assume exist. If you get that successful that you cannot use it anymore (have 10% of earth's population as clients) probably you can train your own models already.

The license prohibits Snap from using it. Meta knows that it would be useful to Snap and that they don’t have the resources to train their own model.
Post reply on HN