Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

701–710 of 860 posts

Re: Llama 2

#701

Post by andsoitis »

cudgy wrote: post #295
Miraste wrote: post #253

Earlier quoted context omitted.

They work (most of the time) in Phoenix and SF because they've mapped every single inch of the cities by now and there are no adverse conditions. It's not scalable.

Why is that not scalable? Mapping out two large cities for an experimental project in a few years seems scalable, expand to new cities over time with additional resources.

I think you’re conflating doable and scalable.

Or perhaps my threshhold for “scalable” takes different parameters and weigh these inputs differentfly from you.

Re: Llama 2

#702

Post by gaogao »

visarga wrote: post #519
brucethemoose2 wrote: post #249

Earlier quoted context omitted.

At some point, higher quality tokens will be far more important than more tokens. No telling how much junk is in that 2T. But I wonder if data augmentations could help? For instance, ask LLaMA 70B to reword everything in a dataset, and you can train over the same data multiple times without repeats.

A great idea. If we are at it, why don't we search all topics and then summarise with a LLM? It would be like an AI made wikipedia 1000x times larger indexing all things, concepts and events, or a super knowledge graph. It would create a lot of training data, and maybe add a bit of introspection to the model - it explicitly knows what it knows. Could help reduce hallucinations, learn attribution, ability to recognise…

I have this pet proposal that LLMs would be pretty nice to help fill out WikiData https://friend.computer/jekyll/update/2023/04/30/wikidata-ll..., as the technique of getting LLMs to write queries, instead of directly giving data, has worked really well so far for me.

Re: Llama 2

#703

Post by ericpauley »

metaquestions wrote: post #676

I keep getting this - been trying sporadically over the past couple hours. Anyone else hit this and any way to work around this Resolving download.llamameta.net (download.llamameta.net)... 108.138.94.71, 108.138.94.95, 108.138.94.120, ... Connecting to download.llamameta.net (download.llamameta.net)|108.138.94.71|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2023-07-18 18:02:19 ERROR 403: F…

I had this and requested a new link by filling the form again. It worked.

Re: Llama 2

#706

Post by famouswaffles »

pclmulqdq wrote: post #693
famouswaffles wrote: post #649

Earlier quoted context omitted.

1.You train on the kind of problems you want to solve. you don't report numbers that evaluate performance based on examples it trained on. Datasets will typically have splits, one for training and another for testing. 2. Open ai is capped profit. They are also not a publicly traded company. researchers are researchers regardless of who they work for. Training on test data is especially stupid for commercial applicati…

I am suggesting that OpenAI's main product is "LLM that benchmarks the best." From that point, it is completely illogical not to train on at least some of the test data (or data that is very similar to the test data) so that you can fudge the numbers in your favor. You don't want to go too far, but overfitting a tiny bit will make you look like you have a significant edge. When someone says that your product isn't th…

Yeah and I'm saying I don't believe it.

I don't know what you're talking about. GPT-4 is the best model out there by significant margin. That's coming from personal usage not benchmarks. A 10% drop in traffic the first month students are out of school is not "losing users quickly" lol.

ChatGPT didn't gain public use waving benchmarks around. We didn't even know what they were until GPT-4's release. The vast majority of its users know nothing about any of that or care. So your first sentence is just kind of nonsensical.

Anyway whatever. If that's what you believe then that's what you believe. Just realize you have nothing to back it up.

Re: Llama 2

#707

Post by poyu »

lucubratory wrote: post #635
nine_k wrote: post #520

Earlier quoted context omitted.

I suppose that current LLMs are incapable of answering such questions by saying "I don't know". The have no notion of facts, or any other epistemic categories. They work basically by inventing a plausible-sounding continuation of a dialog, based on an extensive learning set. They will always find a plausible-sounding answer to a plausible-sounding question: so much learning material correlates to that. Before epistem…

No, that's a common misconception. They do what they are asked to do, and when they are asked to provide an answer they will provide an answer. If you ask them to provide an answer if they know, or tell you that they don't know if they don't know, they will comply with that quite well, and you'll hear a lot of "I don't know"s for questions it doesn't know the answer to.

I think the truth is somewhere in between, since I’ve seen both responses: “I don’t know” and something completely made up that was presented as facts.

Re: Llama 2

#708

Post by airgapstopgap »

foobiekr wrote: post #610
airgapstopgap wrote: post #340

Earlier quoted context omitted.

Do you realize that LLaMA-1 is just a very slightly smaller, comparably performing replication of Chinchilla [1], which DeepMind had completed a year prior to LLaMA's release? And has RLHF-ed into a suitable chatbot "Sparrow" [2] months earlier than ChatGPT was launched? To assume that Google doesn't have anything competitive with Meta is to say that their papers just so happen to contain recipes for Meta's models bu…

Google _internally_ feels that they are way behind. Forget commenters on HN, literally all of the google employees that I know believe that the company is failing here.

This is not responsive to my arguments. Google can be arbitrarily far behind OpenAI or Anthropic, OP's idea that they feel threatened by LLaMA when they (well, Deepmind) have reached LLaMA level 18-10 months ago is still wrong.

Re: Llama 2

#710

Post by famouswaffles »

jacquesm wrote: post #682
famouswaffles wrote: post #640

Earlier quoted context omitted.

a mutation happens and if that mutation succeeds in ensuring survival, it stays and then spreads. Reproduce is a function evolution maximizes for. Not intentionally sure but that's irrelevant. The whole point of artificial neural networks is that they teach themselves. They get an answer wrong, numbers shift and if those numbers help the next instance they stay or shift as needed. There's no intentionality in the shi…

Evolution is not a mechanism that maximizes it is a set of interrelated elements that operate at entirely different levels (molecular, cellular, individual and species) to pass on those traits from one generation to another that result in (possible) change of future individuals within a population, which may affect the survival of those individuals increasing or decreasing the chances of passing those traits on to th…

Whether the brain is "designed" or not is really irrelevant to the point here. It has nothing to do with whether an objective function is "wrong" or not.
Post reply on HN