Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

591–600 of 860 posts

Re: Llama 2

#591

Post by kossTKR »

whimsicalism wrote: post #243
xbmcuser wrote: post #125

Earlier quoted context omitted.

What makes you think that. Apple is the company that would be most successful at hiding something like this then introduce it as siri ai or something. Not that they are I am just saying Apple keeps everything close to its chest when it comes to products it might introduce in the future.

I work in the field and they just are not hiring the people they need to be hiring.

Interesting. The very early adoption of the neural engines in all Apple products would make you think that they had something brewing. Same with the relatively capable m1/2 GPU's. Various models and stable diffusion runs suprisingly fast on these devices and could be optimised to run much, much faster if Apple actually cared, but they weirdly seem not to.

Re: Llama 2

#592

Post by pclmulqdq »

famouswaffles wrote: post #574
gitgud wrote: post #558

Earlier quoted context omitted.

Is it possible that some LLM’s are trained on these benchmarks? Which would mean they’re overfitting and are incorrectly ranked? Or am I misunderstanding these benchmarks?…

Test leakage is not impossible for some benchmarks. But researchers try to avoid/mitigate that as much as possible for obvious reasons.

Given all of the times OpenAI has trained on peoples' examples of "bad" prompts, I am sure they are fine-tuning on these benchmarks. It's the natural thing to do if you are trying to position yourself as the "most accurate" AI.

Re: Llama 2

#593

Post by seydor »

molticrystal wrote: post #482

I asked llama2.ai for some personal advice to see what insights it might offer, it responded: tthtthtthtthtthtth tthtthtthtthtthtth tthtthtthtthtth tthtthtthtthtth tthtthttht tthtthtth tthtth thtth th thtth thtth thtth thtth tth tth tth tthtth tth tth tthtth tthtth tthtth tthtth tthtth ttht tthtth tthtth tthtth tthtth thtthtth thtthtthtth thtthtthtth thtthtth tthtthtth thttht thtthtth thtthtth thtthtth thtth thttht t…

thoughtful

Re: Llama 2

#594

Post by renewiltord »

jll29 wrote: post #579
renewiltord wrote: post #99

Earlier quoted context omitted.

Google's LLMs are all vaporware. No one's ever seen them. They're supposedly mind-blowing but when they are released they always sound like lobotomized monkeys. All the AlphaGo/AlphaFold stuff is very cool, but since no one has seen their LLMs this is about as convincing as my claiming I've donated billions to charity.

I can assure you Google BERT isn't vaporware. It was probably a challenge to integrate it into search, but they did that. So your assertion has been refuted based on your use of "all", at the very least.

Haha, that's right. Google has BERT. Their AI stuff isn't all vaporware. There's always BERT.

Re: Llama 2

#595

Post by cwkoss »

Plugged in a prompt I've been developing for use in a potential product at work (using chatgpt previously).

Llama2 failed pretty hard. "FTP traffic is not typically used for legitimate purposes."

Re: Llama 2

#596

Post by el_nahual »

meindnoch wrote: post #586
el_nahual wrote: post #566

Earlier quoted context omitted.

People keep saying this is commoditize your complement but that's not what this is! Goods A and B are economic complements if, when the price of A goes down, demand for B goes up. LLMs are not complements to social media platforms. There is zero evidence that if "the price of LLMs goes down" then "demand for social media apps go up". This is a case of commoditizing the competition but that's not the same thing. Commo…

>LLMs are not complements to social media platforms Tell that to the people generating text for social media campaigns using LLMs.

Do those campaigns increase or decrease engagement? My gut is that LLM use will decrease social media demand.

Re: Llama 2

#597

Post by lacksconfidence »

cwkoss wrote: post #595

Plugged in a prompt I've been developing for use in a potential product at work (using chatgpt previously). Llama2 failed pretty hard. "FTP traffic is not typically used for legitimate purposes."

Depending on context, thats probably true? i can't think of the last time we preferred ftp over something like scp or rsync. But I could certainly believe some people are still running ancient systems that use ftp.

Re: Llama 2

#598

Post by famouswaffles »

cubefox wrote: post #588
famouswaffles wrote: post #581

Earlier quoted context omitted.

Sparse MoE models are neither new nor secret. The only reason you haven't seen much use of them for LLMs is because they would typically well underperform their dense counterparts. Until this paper ( https://arxiv.org/abs/2305.14705 ) indicated they apparently benefit far more from Instruct tuning than dense models, it was mostly a "good on paper" kind of thing. In the paper, you can see the underperformance i'm talk…

This paper came out well after GPT-4, so apparently this was indeed a secret before then.

The user I was replying to was talking about the now and future.

We also have no indication sparse models outperform dense counterparts so it's scale either way.

Re: Llama 2

#599

Post by famouswaffles »

pclmulqdq wrote: post #592
famouswaffles wrote: post #574

Earlier quoted context omitted.

Test leakage is not impossible for some benchmarks. But researchers try to avoid/mitigate that as much as possible for obvious reasons.

Given all of the times OpenAI has trained on peoples' examples of "bad" prompts, I am sure they are fine-tuning on these benchmarks. It's the natural thing to do if you are trying to position yourself as the "most accurate" AI.

Assuming they were doing that, Fine-tuning on benchmarks isn't the same as test leakage/testing on training data. No researcher is intentionally training on test data.

If it performs about as well in instances it has never seen before (test set) then it's not overfit to the test.

Post reply on HN