Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

341–350 of 860 posts

Re: Llama 2

#341

Post by TaylorAlexander »

jug wrote: post #320
mike_d wrote: post #305

Earlier quoted context omitted.

You're just being overly pedantic. They hold their breath, fully submerge, control their buoyancy, and propel themselves through water. Also known as swimming.

Nah, this is often not considered swimming in major publications and by zoos. National Geographic https://www.nationalgeographic.com/animals/mammals/facts/hip... > Hippos cannot swim or breathe underwater, and unlike most mammals they are so dense that they cannot float. Instead, they walk or run along the bottom of the riverbed. Because their eyes and nostrils are located on the top of their heads, they can still se…

The thing about being "pedantic" with LLMs is this is actually a very good thing to do! These systems output very plausible sounding text and we can trick ourselves in to believing these things behave a like a human, but they fail in subtle ways that we need to be very careful of. Finding examples where it is subtly wrong is very important to understanding what we should expect of these models. If we ignore subtle errors, we could make the mistake of deploying these in situations where those subtle errors can get people hurt. I was just listening to the TWML podcast last night with a Microsoft researcher who studies LLMs, and this "pedantic" analysis is exactly the kind of thing professional LLM engineers concern themselves with! They do all kinds of manipulation and then run standardized tests to see exactly how they perform, because they would like to use these for general purpose computing tasks and subtle errors are exactly the kind of thing they want to eliminate!

It's funny to see people complaining about this when this kind of analysis is very important to improving these systems. I want to remind people not to romanticize these systems or get attached to our expectations of what we think it should be doing. They are a novel computer system and we should be happy to objectively analyze their performance.

Re: Llama 2

#342

Post by mindcrime »

jahewson wrote: post #116
whimsicalism wrote: post #4

Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…

Is there a good name for this kind of license? If not I propose “de-FANGed”.

I've heard people use the term Fauxpen Source[1].

[1]: https://opensource.com/article/19/4/fauxpen-source-bad-busin...

Re: Llama 2

#343

Post by DeathArrow »

galaxyLogic wrote: post #106
lacker wrote: post #78

Earlier quoted context omitted.

I think Google or Microsoft probably would want to use LLaMa for various purposes like benchmarking and improving their own products. Check out this other condition from the license: v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof). https://github.com/facebookresearch/llama/blob/main/LICENS…

I can see their business logic but isn't it a bit like do not allow people (or bots) talk to each other, they might all get smarter. I understand trade-secrets are not free-speech but if the goal is to build better AI to serve the humanity the different bots should learn from each other. They should also criticize each other to find flaws in their thinking and biases.

>but if the goal is to build better AI to serve the humanity

Whose goal is that?

Re: Llama 2

#344

Post by isoprophlex »

jug wrote: post #275
rajko_rad wrote: post #18

Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…

Still fails my hippo test! > Yes, hippos are excellent swimmers. They spend most of their time in the water, where they feed on aquatic plants and escape the heat of the savannah. In fact, hippos are one of the best swimmers among all land mammals. But that's fine. Most do. Hippos don't swim. They walk or hop/skip at best underwater.

Heh I have my own, the "french palindrome" test.

> User: please reverse the following string: "élu par cette crapule". For example, "abc def" should become "fed cba".

> Assistant: "par cette crapule elu"

ChatGPT actually understands what I mean but still fails sometimes, outputting things like it's having a stroke, "eluu elule par cettep raprap"

Re: Llama 2

#345

Post by andrewmutz »

symlinkk wrote: post #136
_b wrote: post #74

Making advanced LLMs and releasing them for free like this is wonderful for the world. It saves a huge number of folks (companies, universities & individuals) vast amount of money and engineering time. It will enable many teams to do research and make products that they otherwise wouldn't be able to. It is interesting to ponder to what extent this is just a strategic move by Meta to make more money in the end, but wh…

“Wonderful for the world” = “I can exploit this to make money”

Most financial transactions happen because the thing that is being bought improves the life of the buyer.

Re: Llama 2

#346

Post by freedomben »

ipaddr wrote: post #335
freedomben wrote: post #264

Earlier quoted context omitted.

I don't think that's even possible, but if it was it would be a disaster because humans don't work that way. We respond to incentive. When giving to charity, the incentive can be as simple as "I feel good" but it's still an incentive.

Some do what's right even if it doesn't feel good. The best charity can be painful.

Definitely, but the pain was decided to be worth it because the feeling the person got from doing the thing (or the pain they avoided by doing the thing) was worth it. For example a friend of mine has an adult kid who is homeless. They have tried to help many times but this person keeps falling back into the same things that led to the situation in the first place. The pain of watching your child go through something can be an incentive to help. My friend has helped his son even though my friend isn't well off and experiences financial pain. He's still responding to an incentive (that of not wanting to watch his child suffer) even though he's experiencing pain from it.

If a person believes they are doing the right thing, their brain will release dopamine (aka a reward) simply for acting within their belief system.

To be clear, I'm not trying to minimize or dismiss people's sacrifices. I still think they are noble

Re: Llama 2

#347

Post by Minor49er »

jug wrote: post #275
rajko_rad wrote: post #18

Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…

Still fails my hippo test! > Yes, hippos are excellent swimmers. They spend most of their time in the water, where they feed on aquatic plants and escape the heat of the savannah. In fact, hippos are one of the best swimmers among all land mammals. But that's fine. Most do. Hippos don't swim. They walk or hop/skip at best underwater.

I asked it about cannibals. It said

> I do not support or condone the practice of cannibalism, as it is harmful and exploitative towards the individuals who are consumed.

Then it said that cannibals have inherent worth and dignity and that we should strive to appreciate what they do. Then it crashed and is now responding to all following inputs with just the letter "I"

Re: Llama 2

#348

Post by andsoitis »

CSMastermind wrote: post #332
jug wrote: post #275

Earlier quoted context omitted.

Still fails my hippo test! > Yes, hippos are excellent swimmers. They spend most of their time in the water, where they feed on aquatic plants and escape the heat of the savannah. In fact, hippos are one of the best swimmers among all land mammals. But that's fine. Most do. Hippos don't swim. They walk or hop/skip at best underwater.

There's a few prompts that I use with every model to compare them. One of the simplest ones is: > When does the bowl of the winds get used in the wheel of time books? LLaMA2 fails pretty hard: > The Bowl of the Winds is a significant artifact in the Wheel of Time series by Robert Jordan. It is first introduced in the third book, "The Dragon Reborn," and plays a crucial role in the series throughout the rest of the bo…

> get this question correct

I am willing to bet a million dollars that it is unlikely any single model will ever be able to answer any question correctly.

The implications then are that one cannot use a single question evaluate whether a model is useful or not.

Re: Llama 2

#349

Post by andybak »

jug wrote: post #337
gpm wrote: post #304

Earlier quoted context omitted.

This test seems to be testing the ability of it to accurately convey fine details about the world. If that's what you're looking for it's a useful test, but if you're looking for a language model and not a general knowledge model I'm not sure it's super relevant. The average person probably couldn't tell you if a hippo swims either, or having been informed about how a hippo locomotes whether or not that counts as swi…

So it's more designed for a superficial chat?

False dichotomy alert

Re: Llama 2

#350

Post by seanthemon »

Minor49er wrote: post #347
jug wrote: post #275

Earlier quoted context omitted.

Still fails my hippo test! > Yes, hippos are excellent swimmers. They spend most of their time in the water, where they feed on aquatic plants and escape the heat of the savannah. In fact, hippos are one of the best swimmers among all land mammals. But that's fine. Most do. Hippos don't swim. They walk or hop/skip at best underwater.

I asked it about cannibals. It said > I do not support or condone the practice of cannibalism, as it is harmful and exploitative towards the individuals who are consumed. Then it said that cannibals have inherent worth and dignity and that we should strive to appreciate what they do. Then it crashed and is now responding to all following inputs with just the letter "I"

Don't look behind your computer..
Post reply on HN