Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

731–740 of 860 posts

Re: Llama 2

#731

Post by henriquez »

bg24 wrote: post #696
el_nahual wrote: post #566

Earlier quoted context omitted.

People keep saying this is commoditize your complement but that's not what this is! Goods A and B are economic complements if, when the price of A goes down, demand for B goes up. LLMs are not complements to social media platforms. There is zero evidence that if "the price of LLMs goes down" then "demand for social media apps go up". This is a case of commoditizing the competition but that's not the same thing. Commo…

Nice analogy and explanation. Another aspect is building a ubiquitous platform and figure out how to monetize later as they (Meta) already have a cash cow. Zuck is a smart leader. Metaverse was a debacle. But the new world (AI centric) is for real. He is likely focusing on both weakening the stronghold of Google and building a massive community (like Android) around llama. Product ideas (including enterprise focus) w…

“AI centric world” is as fake as the fully self-driving car tech that is largely based on the same fundamental concepts and never panned out, even a half decade the investor/speculation hypetrain went off the rails. Dogecoin is more real than so-called AI.

Re: Llama 2

#732

Post by pclmulqdq »

famouswaffles wrote: post #718
pclmulqdq wrote: post #712

Earlier quoted context omitted.

Nobody has any evidence here. I'm saying that the incentives are such that the null hypothesis should be the opposite of what you think.

Your entire argument, Your incentives hinge on "OpenAI's main product is "LLM that benchmarks the best."" which is a particularly silly assertion when Open AI did not release benchmark evaluatios for 3.5 for months. Not when the product was released. Not even when the API was released.

You don't have to release official numbers to run benchmarks. You also don't have to own the LLM to run benchmarks. Within hours of GPT-4's emergence, many benchmarks had been run.

Re: Llama 2

#733

Post by clarge1120 »

nightski wrote: post #643
famouswaffles wrote: post #599

Earlier quoted context omitted.

Assuming they were doing that, Fine-tuning on benchmarks isn't the same as test leakage/testing on training data. No researcher is intentionally training on test data. If it performs about as well in instances it has never seen before (test set) then it's not overfit to the test.

I'm confused, fine-tuning is training. How is that not leakage? I'm hesitant to call them researchers, they are employees of a for-profit company trying to meet investor expectations.

Besides, OpenAI dropped all pretense of being open and transparent as soon as they saw how popular their open and transparent technology had become.

Re: Llama 2

#734

Post by galaxyLogic »

DeathArrow wrote: post #343
galaxyLogic wrote: post #106

Earlier quoted context omitted.

I can see their business logic but isn't it a bit like do not allow people (or bots) talk to each other, they might all get smarter. I understand trade-secrets are not free-speech but if the goal is to build better AI to serve the humanity the different bots should learn from each other. They should also criticize each other to find flaws in their thinking and biases.

>but if the goal is to build better AI to serve the humanity Whose goal is that?

Google's. Do no evil they say

Re: Llama 2

#735

Post by villgax »

ilaksh wrote: post #464
villgax wrote: post #267

Earlier quoted context omitted.

Exactly- You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof).

Ok but that means that theoretically you could actually use Llama 2 to create fine tuning data for Llama 2 right?

Yeah you could do that, I'm just baffled that these guys think they'll be able to monitor something like this being used to augment anything for that matter

Re: Llama 2

#736

Post by famouswaffles »

pclmulqdq wrote: post #732
famouswaffles wrote: post #718

Earlier quoted context omitted.

Your entire argument, Your incentives hinge on "OpenAI's main product is "LLM that benchmarks the best."" which is a particularly silly assertion when Open AI did not release benchmark evaluatios for 3.5 for months. Not when the product was released. Not even when the API was released.

You don't have to release official numbers to run benchmarks. You also don't have to own the LLM to run benchmarks. Within hours of GPT-4's emergence, many benchmarks had been run.

You said their main product was "LLMs that benchmark the best" like benchmarking was some important aspect of marketing. It's not. That's fact. You can't say it's this hugely important thing and conveniently leave out they make near zero effort to do anything with it.

Basically the only people running benchmarks that could have been gamed on GPT-4 were other researchers, not companies, customers or users looking to use a product.

Normal users are certainly not running benchmarks and companies running benchmarks are running ones on internal data, which just defeats the whole point of gaming these research benchmarks.

Re: Llama 2

#737

Post by frabcus »

forrestthewoods wrote: post #654
RobotToaster wrote: post #41

Another AI model pretending to be open source, when it's licence violates point 5 and 6 of the open source definition.

I genuinely have no idea what N-Point definition of open source you’re using. The term “open source” doesn’t have a singular definition. I liked the comment somewhere in this thread that if you stuck 5 HN users in a room you’d get 12 definitions for open source. Sounds like people need to come with more precise terms like “GNU Open Source” or similar. Because at this point we’ve gone too far and there will never be a…

This was a huge thing in the 1990s - yes there is a singular definition, by the Open Source Initiative https://opensource.org/

That's a good thing, because otherwise corporations constantly try to stretch the definition and make it meaningless. Same then, same now!

Re: Llama 2

#738

Post by frabcus »

drexlspivey wrote: post #475
gentleman11 wrote: post #130

Earlier quoted context omitted.

It's not even remotely open source

How do you remotely open source a binary blob? Do you want them to post their training code and dataset?

Interestingly Open Source initiative have a project prominently on their home page to get input for definiing AI open source: https://opensource.org/

Re: Llama 2

#739

Post by lumost »

Thanks be to open-source

https://huggingface.co/models?sort=trending&search=thebloke%...

Has the quantized weights, available to download now. I tried out the Llama-2-7B-GPTQ on an A100 hosted at runpod.io. Llama-2 is anecdotally much better at instruction following for langchain compared to Falcon-7b-GPTQ - but worse than GPT-3.5 and much worse than GPT-4. Specifically, the Llama-2 model is actually capable of using langchain without hitting parse errors. Something that Falcon wasn't capable of.

Would love to hear folks inference setups, the A100 was... not fast - but I didn't spend any time trying to make it fast.

Re: Llama 2

#740

Post by lumost »

fbdab103 wrote: post #714
jasonfarnon wrote: post #629

Earlier quoted context omitted.

You don't see an engineer at an extremely PR-conscious company at least checking how their model performs on popular benchmarks before rolling it out? And if its performance is lackluster, you do you really see them doing nothing about it? It probably doesn't make a huge difference anyway. I know those old vision models were overfitted to the standard image library benchmarks, but they were still very impressive.

Famously, some of the image models were so overtrained they could still yield impressive results if the colors were removed.

This wasn't so much overtraining, as the models learning something different than what we expected. If you look at a pixel by pixel representation of an image, textures tend to be more significant/unique patterns than shapes. There are some funny studies from the mid 2010s exploring this.
Post reply on HN