Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

751–760 of 860 posts

Re: Llama 2

#751

Post by reacharavindh »

stu2b50 wrote: post #19
minimaxir wrote: post #14

Earlier quoted context omitted.

That's an oddly high number for blocking competition. OpenAI's ChatGPT hit 100 million MAUs in January, and has gone down since. It's essentially a "Amazon and Google don't use this k thx."

I think more Apple. It's not like Google or Microsoft would want to use LLaMA when they have fully capable models themselves. I wouldn't be surprised if Amazon does as well. Apple is the big laggard in terms of big tech and complex neural network models.

Considering how much Apple likes to retain control, I’m almost sure they won’t want to use someone else’s model even if it were free in every sense of the word.

Re: Llama 2

#752

Post by mdale »

henriquez wrote: post #731
bg24 wrote: post #696

Earlier quoted context omitted.

Nice analogy and explanation. Another aspect is building a ubiquitous platform and figure out how to monetize later as they (Meta) already have a cash cow. Zuck is a smart leader. Metaverse was a debacle. But the new world (AI centric) is for real. He is likely focusing on both weakening the stronghold of Google and building a massive community (like Android) around llama. Product ideas (including enterprise focus) w…

“AI centric world” is as fake as the fully self-driving car tech that is largely based on the same fundamental concepts and never panned out, even a half decade the investor/speculation hypetrain went off the rails. Dogecoin is more real than so-called AI.

Was this response generated by AI ?

Re: Llama 2

#753

Post by Roark66 »

lappa wrote: post #515

Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…

I have to say in my experience falcon-40b-instruct got very close to chatgpt (gpt-3. 5),even surpassing it in few domains. However, it is important to note (not at all)OpenAI are doing tricks with the model output. So comparing OS models with just greedy output decoding (very simple) is not fair for OS models.

Still, I'm very excited this model at 13B seems to be matching falcon-40B in some benchmarks. I'm looking forward to using it :-)

Re: Llama 2

#754

Post by Roark66 »

I can't currently download the model, can people comment on the architecture. Is it a decoder only model? Is it using multi query attention? How many attention heads in all variants, how big is the context window? I wish they just published these details.

Re: Llama 2

#755

Post by Roark66 »

doctoboggan wrote: post #555
lappa wrote: post #515

Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…

Good to see these results, thanks for posting. I wonder if GPT-4's dominance is due to some secret sauce or if its just the first mover advantage and Llama will be there soon.

In chatgpt there is plenty of "secret sauce" in their output sampling, sending the output for scoring by another model.

As for Gpt4, allegedly it is a combined model(many domain specific models) so perhaps add extra input processing by yet another model to detect problem domain and send it to the right specialised model.

Re: Llama 2

#756

Post by fnl »

Roark66 wrote: post #753
lappa wrote: post #515

Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…

I have to say in my experience falcon-40b-instruct got very close to chatgpt (gpt-3. 5),even surpassing it in few domains. However, it is important to note (not at all)OpenAI are doing tricks with the model output. So comparing OS models with just greedy output decoding (very simple) is not fair for OS models. Still, I'm very excited this model at 13B seems to be matching falcon-40B in some benchmarks. I'm looking fo…

> OpenAI are doing tricks with the model output

Do you have any pointers to the “tricks” that are being applied?

Re: Llama 2

#757

Post by snowstormsun »

lumost wrote: post #695
FanaHOVA wrote: post #680

Earlier quoted context omitted.

Presented with no comment :) https://twitter.com/chhillee/status/1635790330854526981?s=46...

Having worked on ML products, there is sometimes debate on whether you should train on the test partition prior to prod deployment - after all, why would you ship a worse model to prod? Obviously you can't tell whether the model is better at generalization compared to an alternate technique, and you also incur some overfit risk. But many industrial problems are solvable through memorization.

Why would you want to ship an untested model? That's insane.

Re: Llama 2

#758

Post by sundarurfriend »

FanaHOVA wrote: post #680
gitgud wrote: post #558

Earlier quoted context omitted.

Is it possible that some LLM’s are trained on these benchmarks? Which would mean they’re overfitting and are incorrectly ranked? Or am I misunderstanding these benchmarks?…

Presented with no comment :) https://twitter.com/chhillee/status/1635790330854526981?s=46...

Nitter link: https://nitter.net/chhillee/status/1635790330854526981/

Re: Llama 2

#759

Post by rossy »

echelon wrote: post #402
brucethemoose2 wrote: post #176

Earlier quoted context omitted.

Is a truly open source 2 trillion token model even possible? Even if Meta released this under Apache 2.0, there's the sticky question of the training data licenses.

> Even if Meta released this under Apache 2.0, there's the sticky question of the training data licenses. You need to pay Disney royalties out of every paycheck because you watched Toy Story and Star Wars. These movies updated your own neural weights.

This is always such a reductive position. There's a little bit of evidence that ANNs learn and create in a similar way to biological neural networks, and a lot of evidence that they're quite alien to us, including being way better than the average human at reproducing parts of their training set verbatim (eg. Copilot reproducing `Q_rsqrt` verbatim, including the magic numbers, formatting and comments.) So it's not obvious that training an ANN should be treated the same as a human watching a movie under copyright law, which is designed to protect the human creative process only.

Re: Llama 2

#760

Post by zirgs »

whimsicalism wrote: post #133
samwillis wrote: post #103

Earlier quoted context omitted.

Apple would absolutely not want to use a competitors, or any other, public LLM. They want to own the whole stack, and will want to have their own secret source as part of it. It's not like they don't have the capital to invest in training...

Apple does not have the capability to train a LLM currently.

Why not? They have cash and they can rent a bunch of GPUs from Amazon.
Post reply on HN