Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

771–780 of 965 posts

Re: Meta Llama 3

#771

Post by littlestymaar »

jiggawatts wrote: post #667
littlestymaar wrote: post #486

Earlier quoted context omitted.

> It’s a bit like arguing that Linux is not open source because you don’t have every email Linus and the maintainers ever received. Or that you don’t know what lectures Linus attended or what books he’s read. Linux is open source, because you can actually compile it yourself! You don't need Linus's email for that (and if you needed some secret cryptographic key on Linus' laptop to decrypt and compile the kernel, then…

> the same way you can edit the binary of a proprietary game to disable DRMs, that doesn't make it open-source either This is where I have to disagree. Continuing the training of an open model is the same process as the original training run. It's not a fundamentally different operation.

> Continuing the training of an open model is the same process as the original training run. It's not a fundamentally different operation.

In practice it's not (because LoRA) but that doesn't matter: continuing the training is just a patch on top of the initial training, it doesn't matter if this patch is applied through gradient descent as well, you are completely dependent on how the previous training was done, and your ability to overwrite the model's behavior is limited.

For instance, Meta could backdoor the model with specially crafted group of rare tokens to which the model would respond a pre-determined response (say “This is Llama 3 from Meta” as some kind of watermark), and you'd have no way to figure out and get rid of it during fine-tuning. This kind of things does not happen when you have access to the sources.

Re: Meta Llama 3

#772

Post by wiz21c »

CuriouslyC wrote: post #38
bbig wrote: post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

They didn't compare against the best models because they were trying to do "in class" comparisons, and the 70B model is in the same class as Sonnet (which they do compare against) and GPT3.5 (which is much worse than sonnet). If they're beating sonnet that means they're going to be within stabbing distance of opus and gpt4 for most tasks, with the only major difference probably arising in extremely difficult reasonin…

"within stabbing distance"

dunno if english is your mother tongue, but this sounds really good (although a tad aggressive :-) )) !

Re: Meta Llama 3

#773

Post by danielhanchen »

CuriouslyC wrote: post #38
bbig wrote: post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

They didn't compare against the best models because they were trying to do "in class" comparisons, and the 70B model is in the same class as Sonnet (which they do compare against) and GPT3.5 (which is much worse than sonnet). If they're beating sonnet that means they're going to be within stabbing distance of opus and gpt4 for most tasks, with the only major difference probably arising in extremely difficult reasonin…

On the topic of LoRAs and finetuning, have a Colab for LoRA finetuning Llama-3 8B :) https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe...

Re: Meta Llama 3

#774

Post by vineyardmike »

LrnByTeach wrote: post #610
bbig wrote: post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

Losers & Winners from Llama-3-400B Matching 'Claude 3 Opus' etc.. Losers: - Nvidia Stock : lid on GPU growth in the coming year or two as "Nation states" use Llama-3/Llama-4 instead spending $$$ on GPU for own models, same goes with big corporations. - OpenAI & Sam: hard to raise speculated $100 Billion, Given GPT-4/GPT-5 advances are visible now. - Google : diminished AI superiority posture Winners: - AMD, intel: th…

I also disagree on Google...

Google's business is largely not predicated on AI the way everyone else is. Sure they hope it's a driver of growth, but if the entire LLM industry disappeared, they'd be fine. Google doesn't need AI "Superiority", they need "good enough" to prevent the masses from product switching.

If the entire world is saturated in AI, then it no longer becomes a differentiator to drive switching. And maybe the arms race will die down, and they can save on costs trying to out-gun everyone else.

Re: Meta Llama 3

#775

Post by tanelpoder »

MeImCounting wrote: post #738
llm_trw wrote: post #717

Earlier quoted context omitted.

You can build a $6,000 machine with 12 channels DDR5 memory that's big enough to hold an 8bit quantized model. The generation speed is abysmal of course. Anything better than that starts at 200k per machine and goes up from there. Not something you can run at home, but definitely within the budget of most medium sized firms to buy one.

You can build a machine that can run 70b models at great TpS speeds for around 30-60k. That same machine could almost certainly run a 400b model with "useable" speeds. Obviously much slower than current ChatGPT speeds but still, that kind of machine is well within the means of wealthy hobbyists/highly compensated SWEs and small firms.

I just tested llama3:70b with ollama on my old AMD ThreadRipper Pro 3965WX workstation (16-core Zen4 with 8 DDR4 mem channels), with a single RTX 4090.

Got 3.5-4 tokens/s, GPU compute was <20% busy (~90W) and the 16 CPU cores / 32 threads were about 50% busy.

Re: Meta Llama 3

#776

Post by latexr »

eru wrote: post #316
swatcoder wrote: post #271

Earlier quoted context omitted.

> his contributions to ... raising salaries It's fun to be able to retire early or whatever, but driving software engineer salaries out of reach of otherwise profitable, sustainable businesses is not a good thing. That just concentrates the industry in fewer hands and makes it more dependent on fickle cash sources (investors, market expansion) often disconnected from the actual software being produced by their teams.…

> It's fun to be able to retire early or whatever, but driving software engineer salaries out of reach of otherwise profitable, sustainable businesses is not a good thing. That argument could apply to anyone who pays anyone well. Driving up market pay for workers via competition for their labour is exactly how we get progress for workers. (And by 'treat well', I mean the whole package. Fortunately, or unfortunately,…

> Fortunately, or unfortunately, that has the side effect of eg paying veterinary nurses peanuts, because there's always people willing to do those kinds of 'cute' jobs.

Veterinaries (including technicians) have an absurdly high rate of suicide. They have a stressful job, constantly around death and mistreatment situations, and don’t get the respect (despite often knowing more than human doctors) or the salaries to match.

Calling these jobs “cute” or saying the veterinary situation is “fortunate” borders on cruel, but I believe you were just uninformed.

Re: Meta Llama 3

#777

Post by schleck8 »

sunaookami wrote: post #315
schleck8 wrote: post #131

Earlier quoted context omitted.

Just use the Replicate demo instead, you can even alter the inference parameters https://llama3.replicate.dev/ Or run a jupyter notebook from Unsloth on Colab https://huggingface.co/unsloth/llama-3-8b-bnb-4bit

This version doesn't have web search and the image creation though.

The image creation isn't Llama 3, it's not multimodal yet. And the web search is Google and Bing API calls so just use Copilot or Perplexity.

Re: Meta Llama 3

#778

Post by bricee98 »

acchow wrote: post #755
eigenvalue wrote: post #248

Earlier quoted context omitted.

For sure. I just started watching the new Dwarkesh interview with Zuck that was just released ( https://t.co/f4h7ko0M7q ) and you can just tell from the first few minutes that he simply has a different level of enthusiasm and passion and level of engagement than 99% of big tech CEOs.

I've never heard of this person, but many of the questions he asks Zuck show a total lack of any insight in this field. How did this interview even happen?

I actually think Dwarkesh is usually pretty good - this interview wasn’t his best (maybe he was a bit nervous because it’s Zuck?) but his show has had a lot of good conversations that get more into the weeds than other shows in my experience

Re: Meta Llama 3

#779

Post by jimnotgym »

jokethrowaway wrote: post #762
viraptor wrote: post #579

Earlier quoted context omitted.

It's trivial to comply with EU privacy regulation if you're not depending on selling customer data. But if you say "It's because of regulations!" I hope you have a source to back that up.

It is because of regulations. Nothing is trivial and anything has a cost. Not only it impacts existing businesses, it also make it harder for a struggling new business to compete with the current leaders. Regulations in the name of the users are actually just made to solidify the top lobbyists in their positions. The reasons I hate regulations is not because billionaires have to spend an extra week on some employee's…

Do you find EU MOSS harder to deal with that US sales tax?

MOSS is a massive reduction in overhead vs registering in each individual country, isn't it? Or are you really just saying you don't like sales tax?

Re: Meta Llama 3

#780

Post by airspresso »

sen wrote: post #761
segmondy wrote: post #646

Earlier quoted context omitted.

so run it locally, local version is not guarded

My locally-hosted llama3 actually craps itself if I ask it to answer in other languages. It's pretty hilarious. Has been working flawlessly (and impressively fast) for everything in English, then does hilarious glitches in other languages. Eg right now to show it here, I say "Write me a poem about a digital pirate in Danish": Digitalen Pirat På nettet sejler han, En digital pirat, fri og farlig. Han har øjnene på de…

Trying the same on https://llama3.replicate.dev/ with Llama 3-70B gives a perfectly fine response with a long poem in Danish. And then it even translates it to English before concluding the response.
Post reply on HN