Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

601–610 of 860 posts

Re: Llama 2

#601

Post by moonchrome »

echelon wrote: post #402
brucethemoose2 wrote: post #176

Earlier quoted context omitted.

Is a truly open source 2 trillion token model even possible? Even if Meta released this under Apache 2.0, there's the sticky question of the training data licenses.

> Even if Meta released this under Apache 2.0, there's the sticky question of the training data licenses. You need to pay Disney royalties out of every paycheck because you watched Toy Story and Star Wars. These movies updated your own neural weights.

If you recreated those works from memory yeah you would be subject to copyright. There's a whole set of rules around fair use and derivative work.

Re: Llama 2

#602

Post by toomanydoubts »

galaxyLogic wrote: post #106
lacker wrote: post #78

Earlier quoted context omitted.

I think Google or Microsoft probably would want to use LLaMa for various purposes like benchmarking and improving their own products. Check out this other condition from the license: v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof). https://github.com/facebookresearch/llama/blob/main/LICENS…

I can see their business logic but isn't it a bit like do not allow people (or bots) talk to each other, they might all get smarter. I understand trade-secrets are not free-speech but if the goal is to build better AI to serve the humanity the different bots should learn from each other. They should also criticize each other to find flaws in their thinking and biases.

The goal is to build better AI to make more money.

Re: Llama 2

#603

Post by layoric »

whimsicalism wrote: post #241
NotAFood wrote: post #153

Earlier quoted context omitted.

Apple has shown time and time again that they have the human capital and money to tackle massive projects discretely. It's already fairly well known that Apple's NLP experts from Siri have been reallocated to some secret project. They are more than capable of training an LLM but given their track record in other segments they probably want to wait for the technology to become more "polished" and give less hallucinate…

I have not seen Apple demonstrate ML depth in their talent nor have I seen signs that they are hiring extensively for NLP depth. They will soon be able to train an LLM because it simply has become commoditized, but they just are not a major player in this space at all.

Their approach is different, they build ML tech that runs on-device, so whatever they developed has to be able to run efficiently on iPhone/iPad etc.

I don’t think we will “hear” about Apple using LLMs either way because they will no doubt call it something different like they always have.

Re: Llama 2

#604

Post by spookie »

whimsicalism wrote: post #4

Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…

Come on.. It's completely understandable. Why would they treat giants the same way as everyone else? I don't know what to make out of these responses, it's completely legitimate and in their right to do this. At least they release their stuff.

Re: Llama 2

#605

Post by blackoil »

amelius wrote: post #314
vorpalhex wrote: post #311

Earlier quoted context omitted.

Then your project would no longer be FOSS.

Depends on what you value more. Strict adherence to some definition someone once made up, or sustainable funding of your project. If it bothers you, you can call it FOSS* instead.

> sustainable funding of your project

You can always make a commercial closed source project.

Re: Llama 2

#606

Post by jokethrowaway »

chaxor wrote: post #513
GreedClarifies wrote: post #108

Earlier quoted context omitted.

Hard disagree. Google has made it plainly clear that they don't have anything useable in this space. Bard scores below all other commercial model. Google is getting the asses handed to them, badly. I figured that the code red would whip them into shape but the rot runs deep.

It seems you didn't quite hear the argument. I agree with you that the models Google has released to the public are absolutely worthless. That certainly does not mean they don't have extremely performant models at all however. If you actually have worked in the area of NLP for about 10 years, you would recognize how the work from Deepmind is much more novel and innovative than other groups. OpenAI certainly has great…

Why would they have secret unreleased models?

Surely Google can find another team of code monkeys to whip out a frontend if there is money to be made.

I don't think Google is going to pull back from making some more money.

I think the most likely option is that they have a bunch of talented academics who get paid on time to work on what interest them - but they're the stereotypical large inefficient company and they can't coordinate the effort of productionizing some cool models before the competition.

Re: Llama 2

#607

Post by ineedasername »

lappa wrote: post #515

Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…

When were the GPT-4 benchmarks calculated, on original release or more recently? (curious per the debate about alleged gpt-4 nerfing)

Re: Llama 2

#608

Post by astrange »

brucethemoose2 wrote: post #421
Jackson__ wrote: post #399

Earlier quoted context omitted.

You might be thinking of unofficial LLaMA finetunes such as Alpaca, Vicuna, etc. LLaMA 1 was a base model without any safety features in the model itself.

Base LLaMAv1 would refuse to answer certain questions. It wasn't as aggressive as OpenAI models or the safety aligned finetunes, but some kind of alignment was there.

Normal training content has "alignment". It's not going to instantly be super racist and endorse cannibalism if it's "unaligned".

Re: Llama 2

#609

Post by spookie »

jdkee wrote: post #562
chaxor wrote: post #513

Earlier quoted context omitted.

It seems you didn't quite hear the argument. I agree with you that the models Google has released to the public are absolutely worthless. That certainly does not mean they don't have extremely performant models at all however. If you actually have worked in the area of NLP for about 10 years, you would recognize how the work from Deepmind is much more novel and innovative than other groups. OpenAI certainly has great…

On that front, Google's Gemini sounds interesting. See https://www.tomsguide.com/news/googles-new-gemini-ai-could-b...

It's better to wait and see. Either way, they are scraping everyone and everything. If they can't do it...

Re: Llama 2

#610

Post by foobiekr »

airgapstopgap wrote: post #340
jahewson wrote: post #104

Earlier quoted context omitted.

I’ve been hearing “Google has secret better models” for 7 months now. Maybe some UFOs in the hangers at Moffett Field too?

Do you realize that LLaMA-1 is just a very slightly smaller, comparably performing replication of Chinchilla [1], which DeepMind had completed a year prior to LLaMA's release? And has RLHF-ed into a suitable chatbot "Sparrow" [2] months earlier than ChatGPT was launched? To assume that Google doesn't have anything competitive with Meta is to say that their papers just so happen to contain recipes for Meta's models bu…

Google _internally_ feels that they are way behind. Forget commenters on HN, literally all of the google employees that I know believe that the company is failing here.
Post reply on HN