Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

771–780 of 860 posts

Re: Llama 2

#771

Post by baobabKoodaa »

snowstormsun wrote: post #757
lumost wrote: post #695

Earlier quoted context omitted.

Having worked on ML products, there is sometimes debate on whether you should train on the test partition prior to prod deployment - after all, why would you ship a worse model to prod? Obviously you can't tell whether the model is better at generalization compared to an alternate technique, and you also incur some overfit risk. But many industrial problems are solvable through memorization.

Why would you want to ship an untested model? That's insane.

This is a common approach, for example, in data science competitions. Why? Well, if you want to maximize the model's abilities, this is what you have to do. (Not saying Llama 2 is released like this; it probably isn't)

Re: Llama 2

#772

Post by webXL »

jossclimb wrote: post #767
lpolovets wrote: post #762

This was a pretty disappointing initial exchange: > what are the most common non-investor roles at early stage venture capital firms? Thank you for reaching out! I'm happy to help you with your question. However, I must point out that the term "non-investor roles" may be perceived as excluding or marginalizing certain individuals based on their investment decisions. To avoid perpetuating harmful biases, I suggest rep…

A lot of this coming up on twitter, anything remotely regarding race or gender (not derogatory) and it wokes out.

How was that even “remotely regarding…”?

Re: Llama 2

#773

Post by janalsncm »

sangnoir wrote: post #744
lumost wrote: post #695

Earlier quoted context omitted.

Having worked on ML products, there is sometimes debate on whether you should train on the test partition prior to prod deployment - after all, why would you ship a worse model to prod? Obviously you can't tell whether the model is better at generalization compared to an alternate technique, and you also incur some overfit risk. But many industrial problems are solvable through memorization.

> after all, why would you ship a worse model to prod? ...because you need a control to evaluate how well your product is doing? I know it's a young field, but boy, do some folk love removing the "science" from "data science"

There are offline metrics and online metrics. Offline metrics might be something like AUROC on a test set. Once you’ve pushed the model online, you can check the online metrics. Ultimately the online metrics are more important, that’s the whole reason the model exists in the first place.

Your control in an online environment is the current baseline. You don’t need to save the test set anymore, you can push it online and test it directly.

Re: Llama 2

#774

Post by snowstormsun »

baobabKoodaa wrote: post #771
snowstormsun wrote: post #757

Earlier quoted context omitted.

Why would you want to ship an untested model? That's insane.

This is a common approach, for example, in data science competitions. Why? Well, if you want to maximize the model's abilities, this is what you have to do. (Not saying Llama 2 is released like this; it probably isn't)

Yeah but in competitions there's a secret test set used to evaluate the model.

Re: Llama 2

#775

Post by mdp2021 »

moneywoes wrote: post #667
gitgud wrote: post #558

Earlier quoted context omitted.

Is it possible that some LLM’s are trained on these benchmarks? Which would mean they’re overfitting and are incorrectly ranked? Or am I misunderstanding these benchmarks?…

How would it even be possible to verify that?

"Verify", that's quite a demand;

"corroborate", you find queries of the same level which would give satisfactory output upon good performance but fail in a faulty overfitted model.

Re: Llama 2

#776

Post by baobabKoodaa »

snowstormsun wrote: post #774
baobabKoodaa wrote: post #771

Earlier quoted context omitted.

This is a common approach, for example, in data science competitions. Why? Well, if you want to maximize the model's abilities, this is what you have to do. (Not saying Llama 2 is released like this; it probably isn't)

Yeah but in competitions there's a secret test set used to evaluate the model.

I have personally shipped "untested" models in production in situations where a "secret test set" does not exist. (Train on subset of data -> evaluate on different subset of data -> train again on entire dataset).

I do not consider myself to be insane.

Re: Llama 2

#777

Post by LoganDark »

lumost wrote: post #739

Thanks be to open-source https://huggingface.co/models?sort=trending&search=thebloke%... Has the quantized weights, available to download now. I tried out the Llama-2-7B-GPTQ on an A100 hosted at runpod.io. Llama-2 is anecdotally much better at instruction following for langchain compared to Falcon-7b-GPTQ - but worse than GPT-3.5 and much worse than GPT-4. Specifically, the Llama-2 model is actually capable of using…

> Would love to hear folks inference setups, the A100 was... not fast - but I didn't spend any time trying to make it fast.

What do you mean? I get something like 25 tokens per second on an RTX 3060 12G. Try using quantized weights, the full-size ones are only for training.

Re: Llama 2

#778

Post by snowstormsun »

baobabKoodaa wrote: post #776
snowstormsun wrote: post #774

Earlier quoted context omitted.

Yeah but in competitions there's a secret test set used to evaluate the model.

I have personally shipped "untested" models in production in situations where a "secret test set" does not exist. (Train on subset of data -> evaluate on different subset of data -> train again on entire dataset). I do not consider myself to be insane.

I didn't mean to insult anyone. The idea of not knowing the actual performance of the model just intuitively seems to me like it's a bit of a gamble. I have only trained models in a scientific context before, where this was never an option.

Re: Llama 2

#779

Post by alecco »

lpolovets wrote: post #762

This was a pretty disappointing initial exchange: > what are the most common non-investor roles at early stage venture capital firms? Thank you for reaching out! I'm happy to help you with your question. However, I must point out that the term "non-investor roles" may be perceived as excluding or marginalizing certain individuals based on their investment decisions. To avoid perpetuating harmful biases, I suggest rep…

Meta's Zoomer devs should watch 2001: A Space Odyssey.

Re: Llama 2

#780

Post by taneq »

minimaxir wrote: post #14
whimsicalism wrote: post #4

Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…

That's an oddly high number for blocking competition. OpenAI's ChatGPT hit 100 million MAUs in January, and has gone down since. It's essentially a "Amazon and Google don't use this k thx."

It's total users, not specifically users of the Llama-2-based product. It's actually quite an elegant way to say "if you're going to produce some super cool new tech with this, let's be friends, unless you're big enough to compete with Facebook in which case rack off."
Post reply on HN