Llama 3.1 • Hacker News Board

Re: Llama 3.1

#11

Post by primaprashant » Tue, Jul 23, 2024, 2:59 PM UTC

The resources for link to model card[1], research paper, and Prompt Guard Tutorial[2] on the page doesn't exist yet

[1]: https://github.com/meta-llama/llama-models/blob/main/models/...

[2]: https://github.com/meta-llama/llama-recipes/blob/main/recipe...

Re: Llama 3.1

#12

Post by angoragoats » Tue, Jul 23, 2024, 2:59 PM UTC

lolinder wrote: post #8

foxhop wrote: post #3

Earlier quoted context omitted.

Your going to need a lot more than a few, 800G VRAM needed

Quantized to 4 bits you'll only need ~200GB! 5 4090s should cover it.

You'll probably need 9 or more. 4090s have 24GB each.

Re: Llama 3.1

#13

Post by AaronFriel » Tue, Jul 23, 2024, 2:59 PM UTC

foxhop wrote: post #3

TechDebtDevin wrote: post #2

Nice, someone donate me a few 4090s :(

Your going to need a lot more than a few, 800G VRAM needed

If previous quantization results hold up, fp8 will have nearly identical performance while using 405GiB for weights, but the KV cache size will still be significant.

Too bad, too, I don't think my PC will fit 20 4090s (480GiB).

Re: Llama 3.1

#14

Post by meetpateltech » Tue, Jul 23, 2024, 3:00 PM UTC

Open Source AI Is the Path Forward - Mark Zuckerberg

https://about.fb.com/news/2024/07/open-source-ai-is-the-path...

Re: Llama 3.1

#15

Post by albert_e » Tue, Jul 23, 2024, 3:00 PM UTC

this "Model Card" github link on [https://llama.meta.com/docs/overview/] seems broken?

https://github.com/meta-llama/llama-models/blob/main/models/...

Re: Llama 3.1

#16

Post by beeboobaa3 » Tue, Jul 23, 2024, 3:01 PM UTC

foxhop wrote: post #3

TechDebtDevin wrote: post #2

Nice, someone donate me a few 4090s :(

Your going to need a lot more than a few, 800G VRAM needed

how is this even useful? no one can run it.

Re: Llama 3.1

#17

Post by Vagantem » Tue, Jul 23, 2024, 3:01 PM UTC

As someone who just started generating AI landing pages for Dropory, this is music to my ears

Re: Llama 3.1

#18

Post by phyrex » Tue, Jul 23, 2024, 3:01 PM UTC

ThrowawayTestr wrote: post #7

Are there any other models with free unlimited use like chatgpt?

meta.ai

Re: Llama 3.1

#19

Post by [deleted] » Tue, Jul 23, 2024, 3:01 PM UTC

[deleted]

Re: Llama 3.1

#20

Post by [deleted] » Tue, Jul 23, 2024, 3:01 PM UTC

[deleted]