Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

241–250 of 860 posts

Re: Llama 2

#241

Post by whimsicalism »

NotAFood wrote: post #153
whimsicalism wrote: post #133

Earlier quoted context omitted.

Apple does not have the capability to train a LLM currently.

Apple has shown time and time again that they have the human capital and money to tackle massive projects discretely. It's already fairly well known that Apple's NLP experts from Siri have been reallocated to some secret project. They are more than capable of training an LLM but given their track record in other segments they probably want to wait for the technology to become more "polished" and give less hallucinate…

I have not seen Apple demonstrate ML depth in their talent nor have I seen signs that they are hiring extensively for NLP depth.

They will soon be able to train an LLM because it simply has become commoditized, but they just are not a major player in this space at all.

Re: Llama 2

#242

Post by derefr »

herpderperator wrote: post #229
rajko_rad wrote: post #18

Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…

...is this supposed to make sense? https://www.dropbox.com/scl/fi/5j0mtwdrqm08dxbh84hh6/Screen-...

You're expecting a language model (as opposed to a more general form of ML model) to be numerate? It has no idea that the token "$105,000" has a relationship "is less than" with the token "$140,000".

It probably knows that the token "1" has the relationship "is less than" with the token "2" — but that's because it has "1" and "2" as reified concepts, each with many different facts and properties and relationships associated directly with those tokens-as-vertices.

"$105,000", meanwhile, is just a lexeme. It maybe knows, due to pre-parsing, that it's "an amount of dollars" — and maybe it even recognizes its order-of-magnitude. It can therefore likely make any statement that takes the token "$105,000" as a meta-syntactic variable standing in for some unknown "amount of dollars." But there's no little numeric model embedded inside the language model that would tell it how many dollars, or be able to compare dollars against dollars.

Re: Llama 2

#243

Post by whimsicalism »

xbmcuser wrote: post #125
stu2b50 wrote: post #19

Earlier quoted context omitted.

I think more Apple. It's not like Google or Microsoft would want to use LLaMA when they have fully capable models themselves. I wouldn't be surprised if Amazon does as well. Apple is the big laggard in terms of big tech and complex neural network models.

What makes you think that. Apple is the company that would be most successful at hiding something like this then introduce it as siri ai or something. Not that they are I am just saying Apple keeps everything close to its chest when it comes to products it might introduce in the future.

I work in the field and they just are not hiring the people they need to be hiring.

Re: Llama 2

#244

Post by TechBro8615 »

unshavedyak wrote: post #219
ipaddr wrote: post #198

Earlier quoted context omitted.

You act like this is a gift of charity instead of attempts to stay relevant.

The best charity is one that benefits both parties, no?

That's just a trade. If we assume "charity" is "altruism," then by definition there must be no benefit to the giver.

Re: Llama 2

#245

Post by bloaf »

DebtDeflation wrote: post #218
swyx wrote: post #177

Earlier quoted context omitted.

if you red team the 13b and the 70b and they pass, what is the danger of 34B being significantly more dangerous? edit: turns out I should RTFP. there was a ~2x spike in safety violations for 34B https://twitter.com/yacineMTB/status/1681358362057883680?s=2...

A 34B model is probably about the largest you can run on a consumer GPU with 24GB VRAM. 70B will require A100's or a cloud host. 13B models are everywhere already. I'm sure this was a very deliberate choice - let people play with the 13B model locally to whet their appetite and then they can pay to run the 70B model on Azure.

I'm running a 30B model on an amd 5600x cpu at 2-3 tokens/s, which is just under a "read-aloud" pace. I'd wager that you can run a 70B model at about the same speed with a 7900x and a bit more RAM.

Re: Llama 2

#246

Post by austinkhale »

Miraste wrote: post #111
chaxor wrote: post #60

Earlier quoted context omitted.

Google has far better models than llama based models. They just simply don't put them facing the public. It is pretty ridiculous that they essentially just set a marketing team with no programming experience to write Bard, but that shouldn't fool anyone into believing they don't have capable models in Google. If Deepmind were to actually provide what they have in some usable form, it would likely be quite good. Despi…

This reminds me of how any day now their self driving cars are going to work right.

Their self driving cars do work? I rode in one for 30 minutes one-way on Sunday. Used it for my return trip too. No driver. Take at least 2 - 3 rides a week and have been for a few months now.

Re: Llama 2

#247

Post by aseipp »

0cf8612b2e1e wrote: post #212
itake wrote: post #109

Can someone reply with the checksums of their download? I will share mine once its finished.

Enormous complaint about this space: people seemingly never think to include checksums. Drives me wild when there is supposedly all of this concern about the right data and provenance, yet it is not easy to even confirm you have the genuine article.

The checksums are automatically included with the models when you download them using the download.sh script, and verified right after the download completes. This isn't unlike how a lot of packages distribute the SHA256SUMS file next to their downloads over HTTPS, which you can validate yourself.

That said it would be nice to announce them somewhere else but if you're already downloading them from Meta directly the need for third party verification is much smaller IMO. Torrents will come soon enough anyway.

Re: Llama 2

#248

Post by fmajid »

DebtDeflation wrote: post #218
swyx wrote: post #177

Earlier quoted context omitted.

if you red team the 13b and the 70b and they pass, what is the danger of 34B being significantly more dangerous? edit: turns out I should RTFP. there was a ~2x spike in safety violations for 34B https://twitter.com/yacineMTB/status/1681358362057883680?s=2...

A 34B model is probably about the largest you can run on a consumer GPU with 24GB VRAM. 70B will require A100's or a cloud host. 13B models are everywhere already. I'm sure this was a very deliberate choice - let people play with the 13B model locally to whet their appetite and then they can pay to run the 70B model on Azure.

Or a $5000 128GB Mac Studio, that you can get for 1/2 the price of a 40GB A100 or 1/7 the price of a 80GB H100.

Re: Llama 2

#249

Post by brucethemoose2 »

joshhart wrote: post #206

From a modeling perspective, I am impressed with the effects of training on 2T tokens rather than 1T. Seems like this was able to get LLAMA v2 7b param models equivalent to LLAMA v1's 13b performance, and the 13b similar to 30b. I wonder how far this can be scaled up - if it can, we can get powerful models on consumer GPUs that are easy to fine tune with QLORA. A RTX 4090 can serve an 8-bit quantized 13b parameter mo…

At some point, higher quality tokens will be far more important than more tokens. No telling how much junk is in that 2T.

But I wonder if data augmentations could help? For instance, ask LLaMA 70B to reword everything in a dataset, and you can train over the same data multiple times without repeats.

Re: Llama 2

#250

Post by seydor »

asdasdddddasd wrote: post #28

Very cool! One question, is this model gimped with safety "features"?

apart from the non-chat model, there are 2 chat models:

> Others have found that helpfulness and safety sometimes trade off (Bai et al., 2022a), which can make it challenging for a single reward model to perform well on both. To address this, we train two separate reward models, one optimized for helpfulness (referred to as Helpfulness RM) and another for safety (Safety RM)

Post reply on HN