Llama 3.1

foxhop wrote: post #3

TechDebtDevin wrote: post #2

Nice, someone donate me a few 4090s :(

Your going to need a lot more than a few, 800G VRAM needed

Oof.

Re: Llama 3.1

#6

Post by glitchc » Tue, Jul 23, 2024, 2:55 PM UTC

foxhop wrote: post #3

TechDebtDevin wrote: post #2

Nice, someone donate me a few 4090s :(

Your going to need a lot more than a few, 800G VRAM needed

Christ!!

Re: Llama 3.1

#7

Post by ThrowawayTestr » Tue, Jul 23, 2024, 2:57 PM UTC

Are there any other models with free unlimited use like chatgpt?

Re: Llama 3.1

#8

Post by lolinder » Tue, Jul 23, 2024, 2:58 PM UTC

foxhop wrote: post #3

TechDebtDevin wrote: post #2

Nice, someone donate me a few 4090s :(

Your going to need a lot more than a few, 800G VRAM needed

Quantized to 4 bits you'll only need ~200GB! 5 4090s should cover it.

Re: Llama 3.1

#9

Post by hubraumhugo » Tue, Jul 23, 2024, 2:58 PM UTC

I wrote about this when llama-3 came out, and this launch confirms it:

Meta's goal from the start was to target OpenAI and the other proprietary model players with a "scorched earth" approach by releasing powerful open models to disrupt the competitive landscape.

Meta can likely outspend any other AI lab on compute and talent:

- OpenAI makes an estimated revenue of $2B and is likely unprofitable. Meta generated a revenue of $134B and profits of $39B in 2023.

- Meta's compute resources likely outrank OpenAI by now.

- Open source likely attracts better talent and researchers.

- One possible outcome could be the acquisition of OpenAI by Microsoft to catch up with Meta.

The big winners of this: devs and AI product startups

Re: Llama 3.1

#10

Post by lawlessone » Tue, Jul 23, 2024, 2:58 PM UTC

TechDebtDevin wrote: post #2

Nice, someone donate me a few 4090s :(

maybe someone will figure out some ways to prune/ quantize it a huge amount ;-;

edit: If the AI bubble pops we will be swimming in GPUs... but no new models.