Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

61–70 of 965 posts

Re: Meta Llama 3

#61

Post by refulgentis »

Vt71fcAqt7 wrote: post #47
refulgentis wrote: post #30

Earlier quoted context omitted.

Over yonder: https://x.com/alexalbert__/status/1780707227130863674 my $0.02: it makes me very uncomfortable that people misunderstand LLMs enough to even think this is possible

Of course it is possible. For example via quantization. Unless you are refering to something I can't see in that tweet. (not signed in).

You're right, that's a good point. It is possible to make a model dumber via quantization.

But even F16 -> llama.cpp Q4 (3.8 bits) has negligible perplexity loss.

Theoratically, a leading AI lab could quantize absurdly poorly after the initial release where they know they're going to have huge usage.

Theoratically, they could be lying even though they said nothing changed.

At that point, I don't think there's anything to talk about. I agree both of those things are theoratically possible. But it would be very unusual, 2 colossal screwups, then active lying, with many observers not leaking a word.

Re: Meta Llama 3

#62

Post by swores »

oersted wrote: post #42
trevor-e wrote: post #11

Earlier quoted context omitted.

Why do people keep saying that Claude3 has been nerfed? Their CTO has said on Twitter multiple times that not a single byte has been changed since its launch, so I'm curious why I keep hearing this. edit: having trouble finding the tweet I saw recently, it might have been from their lead engineer and not the CTO.

It's likely true that they didn't change the model, same for the many claims of GPT-4 getting worse. But they do keep iterating a lot on the "safety" layers on top: classifiers to detect dangerous requests, the main system prompt... But I also think it's partially a psychological phenomenon, just people getting used to the magic and finding more bad edge-cases as it is used more. EDIT: It seems that they do claim tha…

While I do think that many claims of GPT4 getting worse were subjective and incorrect, there certainly was an accidental nerfing of at least ChatGPT Plus, as confirmed by OpenAI releasing an update some months ago specifically acknowledging that it had become "more lazy" and the update was to rectify it.

(I think it was just the settings for how ChatGPT calls the GPT4 model, and not affecting use of GPT4 by API, though I may be misremembering.)

Re: Meta Llama 3

#64

Post by CuriouslyC »

swores wrote: post #54
CuriouslyC wrote: post #45

Earlier quoted context omitted.

Beyond that, to people who interact with the models regularly the "nerf" issue is pretty obvious. It was pretty clear when a new model rollout caused ChatGPT4 to try and stick to the "leadup, answer, explanation" response model and also start to get lazy about longer responses.

That's a different company's model, so while it may have been obvious it is not relevant to whether Claude 3 has been nerfed or not is it?

I use claude3 opus daily and I haven't noticed a change in its outputs, I think it's more likely that there's a discontinuity in the inputs the user is providing to claude which is tipping it over a threshold into a response type they find incorrect.

When GPT4 got lobotomized, you had to work hard to avoid the new behavior, it popped up everywhere. People claiming claude got lobotomized seem to be cherry picking example.

Re: Meta Llama 3

#67

Post by observationist »

https://github.com/meta-llama/llama3/blob/main/LICENSE

Llama is not open source. It's corporate freeware with some generous allowances.

Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source."

https://opensource.org/license

Nothing about Meta's license is open source. It's a carefully constructed legal agreement intended to prevent any meaningful encroachment by anyone, ever, into any potential Meta profit, and to disavow liability to prevent reputational harm in the case of someone using their freeware for something embarrassing.

If you use it against the license anyway, you'll just have to hope you never get successful enough that it becomes more profitable to sue you and take your product away than it would be annoying to prosecute you under their legal rights. When the threshold between annoying and profitable is crossed, Meta's lawyers will start sniping and acquiring users of their IP.

Re: Meta Llama 3

#68

Post by jasonjmcghee »

hermesheet wrote: post #36

Lots of great details in the blog: https://ai.meta.com/blog/meta-llama-3/ Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!

It absolutely does not say that. It in fact provides benchmarks that show it under performing them.

Not great to blindly trust benchmarks, but there are no claims it will outperform GPT-4 or Opus.

It was a checkpoint, so it's POSSIBLE it COULD outperform.

Re: Meta Llama 3

#69

Post by vacuumcl »

hermesheet wrote: post #36

Lots of great details in the blog: https://ai.meta.com/blog/meta-llama-3/ Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!

Comparing to the numbers here https://www.anthropic.com/news/claude-3-family the ones of Llama 400B seem slightly lower, but of course it's just a checkpoint that they benchmarked and they are still training further.

Re: Meta Llama 3

#70

Post by throwup238 »

2c2c wrote: post #51

this space moves impossibly fast... are all open models still just derivatives of llama?

> are all open models still just derivatives of llama?

No there are several others but by far the best alongside llama are Mistral’s mistral and mixtral models. Those are called the foundation models which everyone else builds on top of

Post reply on HN