Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

831–840 of 860 posts

Re: Llama 2

#831

Post by wedesoft »

nightski wrote: post #821
baobabKoodaa wrote: post #809

Earlier quoted context omitted.

I have personally never seen a situation where more training data (of similar quality) causes the model to perform worse . Have you seen such a situation? Please provide example. Your suggestion of running 1000 training runs with different subsets of data sounds excessive and unnecessary to me.

You have to know when to stop training. How are you going to do that without a test set? How do you know when you have achieved generalization without over-fitting?

Early stopping is just one way of regularization. You can use L2 or dropout and then you can train until your model converges.

Re: Llama 2

#832

Post by baobabKoodaa »

nightski wrote: post #822
DougBTX wrote: post #813

Earlier quoted context omitted.

Here's another way to look at it. The test set is an approximation for how the model will perform against production data, but the actual performance of the model is how it performs for actual end-users. So real _actual_ results are always unknown util after the fact. Given that, if the metrics from training clearly show that more data == better model, and there's no reason to expect that trend to reverse, then the l…

You act like training is this pre-set process you just "do". That's not the case, you train until you reach desired performance on the test set. If you don't have a test set how do you know when to stop training and avoid overfitting?

You're confusing training epochs with dataset size.

I'm simplifying now, but you can think of epochs as "how many times we train over the entire dataset? 1 time? 10 times?"

Correspondingly, you can think of dataset size as "how many Wikipedia pages we include in the dataset? 1 million? 10 million?"

Now let's think about overfitting.

What happens when you increase epochs is the model is more likely to overfit your data.

What happens when you increase dataset size is the model is less likely to overfit your data.

Re: Llama 2

#833

Post by baobabKoodaa »

nightski wrote: post #821
baobabKoodaa wrote: post #809

Earlier quoted context omitted.

I have personally never seen a situation where more training data (of similar quality) causes the model to perform worse . Have you seen such a situation? Please provide example. Your suggestion of running 1000 training runs with different subsets of data sounds excessive and unnecessary to me.

You have to know when to stop training. How are you going to do that without a test set? How do you know when you have achieved generalization without over-fitting?

Usually I develop models with a train/validation/test split, where I'm measuring results on the validation set to decide the appropriate number of epochs to use. Then I burn the test set to evaluate performance. Then I train from scratch on the entire dataset (no split) and I use the same number of epochs to train here. Is this number of epochs optimal when the dataset is different? Of course not. But when you use regularization and other methods to combat overfitting appropriately, your training is not going to be overly sensitive to changes in epoch number anyway.

Re: Llama 2

#834

Post by cosmojg »

ghayes wrote: post #763
pallas_athena wrote: post #761

>What's the incipit of 1984? 1984 is a dystopian novel written by George Orwell, published in 1949. The incipit of the novel is: "It was a bright cold day in April, and the clocks were striking thirteen." >what does "striking thirteen" mean? "Striking thirteen" is a metaphor used by Orwell to convey the idea that the world in which the story takes place is not our own, but rather a dystopian society where the totalit…

The same (follow up) response from ChatGPT-4: > "Striking thirteen" refers to the 24-hour clock system, also known as military time, where 1 PM is 13:00, 2 PM is 14:00, and so on until 12 midnight is 0:00. > In the context of George Orwell's "1984," the phrase "the clocks were striking thirteen" is significant for a few reasons. First, it introduces a sense of unease or strangeness for readers accustomed to the 12-ho…

I actually prefer Llama 2's response for its conciseness and the fact that analog clocks, even in places which use 24-hour timekeeping, typically count no higher than 12. The existence and nature of "the 24-hour clock system" is irrelevant to explaining the meaning of "striking thirteen" in this particular context.

Put another way, at least in this one-off example, Llama 2 demonstrates more context awareness than GPT-4.

Re: Llama 2

#836

Post by scinerio »

Speaking strictly on semantics, why does open source have to also mean free? I've heard the term "FOSS" for over a decade now, and it very clearly separates the "free" and "open source" parts.

Releasing with this model allows for AI-based creativity while still protecting Meta as a company. I feel like it makes plenty sense for them to do this.

Re: Llama 2

#837

Post by d0mine »

nottheengineer wrote: post #403
arbuge wrote: post #154

Earlier quoted context omitted.

I like the way the playground allows easy modification of the system prompt. I suggest adding "You are very cranky." to the default prompt for interesting results.

Holy shit, I've never seen an AI go schizophrenic this hard. That's my first try: https://gist.github.com/miguel7501/983f794e13cc762eb6274c9b2...

Every digit is equally likely in pi:

    >>> statistics.mean(map(int, string.digits))
    4.5

Re: Llama 2

#838

Post by lacksconfidence »

xyos wrote: post #132
flimflamm wrote: post #107

Seems not be able to use other languages than English. "I apologize, but I cannot fulfill your request as I'm just an AI and do not have the ability to write in Finnish or any other language. "

it replies in Spanish.

it also replies in pig latin and klingon. Sadly the results are completely wrong, but it tries.

Re: Llama 2

#839

Post by cevn »

lucubratory wrote: post #687
cevn wrote: post #684

Earlier quoted context omitted.

Sounds about right now that you mention it. Time for a re read I guess.. :)

As always :P me and my husband will be starting our first WoT reread together (I got him into the books after he loved the show), just as soon as we finish the current series we're on, which is Children of Time by Adrian Tchaikovsky. Absolutely fantastic series.

Hm... might have to check that out as well, I love sci fi too.. Hopefully he likes the books more than the show, hah.

Re: Llama 2

#840

Post by _ea1k »

Matrixik wrote: post #618
_ea1k wrote: post #441

Earlier quoted context omitted.

While that is true, I'm not aware of any model that has been trained to do that. And all models can do is to do what they were trained to do.

It's not model but working on top of it: https://www.phind.com/ It's asking clarifying questions.

That's actually pretty good. Thanks!
Post reply on HN