Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

581–590 of 860 posts

Re: Llama 2

#581

Post by famouswaffles »

bbor wrote: post #572
famouswaffles wrote: post #567

Earlier quoted context omitted.

It's just scale. But scale that comes with more than an order of magnitude more expense than the Llama models. I don't see anyone training such a model and releasing it for free anytime soon

I thought it was revealed to be fundamentally ensemblamatic in a way the others weren’t? Using “experts” I think? Seems like it would meet the bar for “secret sauce” to me

Sparse MoE models are neither new nor secret. The only reason you haven't seen much use of them for LLMs is because they would typically well underperform their dense counterparts.

Until this paper (https://arxiv.org/abs/2305.14705) indicated they apparently benefit far more from Instruct tuning than dense models, it was mostly a "good on paper" kind of thing.

In the paper, you can see the underperformance i'm talking about.

Flan-Moe-32b(259b total) scores 25.5% on MMLU pre Instruct tuning and 65.4 after.

Flan 62b scores 55% before Instruct tuning and 59% after.

Re: Llama 2

#582

Post by salmo »

jug wrote: post #275
rajko_rad wrote: post #18

Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…

Still fails my hippo test! > Yes, hippos are excellent swimmers. They spend most of their time in the water, where they feed on aquatic plants and escape the heat of the savannah. In fact, hippos are one of the best swimmers among all land mammals. But that's fine. Most do. Hippos don't swim. They walk or hop/skip at best underwater.

I accidentally wandered into a test trying to find something out:

> Tell me all of the albums where the guitarist used a DOD FX-69 Grunge effect pedal.

> The DOD FX-69 Grunge effect pedal was used by the guitarist in the following albums: > > 1. "Nevermind" by Nirvana (1991) > 2. "In Utero" by Nirvana (1993) > 3. "Bleach" by Nirvana (1989) > 4. "Superunknown" by Soundgarden (1994) > 5. "Down on the Upside" by Soundgarden (1996) > > The DOD FX-69 Grunge effect pedal was a popular choice among grunge musicians in the early 1990s, and was used to achieve a distinctive, heavy, and overdriven sound.

They've all failed almost identically. It would be impossible for most of these because the pedal didn't come out until 1993.

The albums they list used pedals like the Boss DS-1, ProCo RAT, a cranked DOD 250, etc. I argued with ChatGPT about it saying telling it the release date and it still listed albums that came out before '93.

Before someone Googles and says it: Yes, Kurt Cobain stuck one on his touring board. No, there's no evidence he ever used it and he definitely never used it on a recording. Most think he just thought it was funny... and it did end up driving sales.

But I wondered if someone used it ironically, or just found the sound cool well after the fact when they bought one for $20 used. I still haven't found an artist using one on a recording. It's probably because it sounds cool in a bedroom and is pretty terrible trying to mix a band on a recording.

Re: Llama 2

#583

Post by [deleted] »

whimsicalism wrote: post #4

Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…

[deleted]

Re: Llama 2

#584

Post by marcopicentini »

lappa wrote: post #515

Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…

How they compare the exact value returned in a response? I found that returning a stable json format is something unpredictable or it reply in a different language.

Re: Llama 2

#585

Post by mirekrusin »

whimsicalism wrote: post #313
PeterStuer wrote: post #258

Earlier quoted context omitted.

Would you believe OpenAI has vastly better models that they are not releasing publicly?

No

GPT models were internally available 6-12 months before they've seen public beta, of course OpenAI has more capable internal models.

Re: Llama 2

#586

Post by meindnoch »

el_nahual wrote: post #566
whimsicalism wrote: post #4

Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…

People keep saying this is commoditize your complement but that's not what this is! Goods A and B are economic complements if, when the price of A goes down, demand for B goes up. LLMs are not complements to social media platforms. There is zero evidence that if "the price of LLMs goes down" then "demand for social media apps go up". This is a case of commoditizing the competition but that's not the same thing. Commo…

>LLMs are not complements to social media platforms

Tell that to the people generating text for social media campaigns using LLMs.

Re: Llama 2

#587

Post by meroes »

avaer wrote: post #148
rajko_rad wrote: post #18

Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…

How does one apply for a job with the the internal A16Z teams experimenting with this?

Ask Llama of course. Showing that you are willing to ask an LLM is a perfect sign for a candidate!

Re: Llama 2

#588

Post by cubefox »

famouswaffles wrote: post #581
bbor wrote: post #572

Earlier quoted context omitted.

I thought it was revealed to be fundamentally ensemblamatic in a way the others weren’t? Using “experts” I think? Seems like it would meet the bar for “secret sauce” to me

Sparse MoE models are neither new nor secret. The only reason you haven't seen much use of them for LLMs is because they would typically well underperform their dense counterparts. Until this paper ( https://arxiv.org/abs/2305.14705 ) indicated they apparently benefit far more from Instruct tuning than dense models, it was mostly a "good on paper" kind of thing. In the paper, you can see the underperformance i'm talk…

This paper came out well after GPT-4, so apparently this was indeed a secret before then.

Re: Llama 2

#589

Post by microtherion »

stOneskull wrote: post #406
rajko_rad wrote: post #18

Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…

> say the tracklist for lou reed album 'new york' >Sure! The tracklist for Lou Reed's album "New York" is as follows: >"Heroin" >"New York City" >"Kicks" >"Betrayed" >"I'm Waiting for the Man" >"The Kids" >"Oh! Sweet Honey Pie" >"Sunday Morning" >"Romeo Had Juliette" >"The Bed" >I hope that helps! Let me know if you have any other questions.

If LLama 2 were a Lou Reed album, it would probably be Metal Machine Music.

Re: Llama 2

#590

Post by raincole »

el_nahual wrote: post #566
whimsicalism wrote: post #4

Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…

People keep saying this is commoditize your complement but that's not what this is! Goods A and B are economic complements if, when the price of A goes down, demand for B goes up. LLMs are not complements to social media platforms. There is zero evidence that if "the price of LLMs goes down" then "demand for social media apps go up". This is a case of commoditizing the competition but that's not the same thing. Commo…

If we're going to theory-crafting, I think if the price of LLMs goes down, the demand for social media should go down too. Cause it's easy to make social media platforms worse with LLMs.
Post reply on HN