Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

611–620 of 965 posts

Re: Meta Llama 3

#611

Post by xyproto »

mensetmanusman wrote: post #588
jph00 wrote: post #160

Earlier quoted context omitted.

> " Nothing about Meta's license is open source. It's a carefully constructed legal agreement intended to prevent any meaningful encroachment by anyone, ever, into any potential Meta profit, and to disavow liability to prevent reputational harm in the case of someone using their freeware for something embarrassing. " You seem to be making claims that have little connection to the actual license. The license states yo…

That's convenient because I only have 699,999,999 customers.

Simultaneously.

Re: Meta Llama 3

#612

Post by elicksaur »

aussieguy1234 wrote: post #554

"You’ll also soon be able to test multimodal Meta AI on our Ray-Ban Meta smart glasses." Now this is interesting. I've been thinking for some time now that traditional computer/smartphone interfaces are on the way out for all but a few niche applications. Instead, everyone will have their own AI assistant, which you'll interact with naturally the same way as you interact with other people. Need something visual? Just…

There are a dozen different services to get the last X days of MSFT stock price. If you’re interested in stocks, you probably have a favorite already. Why would someone need an AI assistant for this?

Re: Meta Llama 3

#613

Post by nbzso »

I still wonder how the smartest people in the world are blind from the inevitable future in which not only cognitive abilities of the masses will decline, but every technology around will be locked and inaccessible. Why this urge for "progress" by big corporations? Is this technology reversible? How is aligned to collective human values? What data is in the training set? Why trust someone without public weights? What will be the value of human knowledge and expertise in this new "economy"? Why target the artistic and creative people first? Who will watch the watchers? Are we collectively lost our minds?

Re: Meta Llama 3

#614

Post by dhon_ »

selcuka wrote: post #594
nathanh4903 wrote: post #582

I tried generating a Chinese rap song, and it did generate a pretty good rap. However, upon completion, it deleted the response, and showed > I don’t understand Chinese yet, but I’m working on it. I will send you a message when we can talk in Chinese. I tried some other languages and the same. It will generate non-English language, but once its done, the response is deleted and replaced with the message

I'm seeing the same behaviour. It's as if they have a post-processor that evaluates the quality of the response after a certain number of tokens have been generated, and reverts the response if it's below a threshold.

I've noticed Gemini exhibiting similar behaviour. It will start to answer, for example, a programming question - only to delete the answer and replace it with something along the lines of "I'm only a language model, I don't know how to do that"

Re: Meta Llama 3

#615

Post by murderfs »

dom96 wrote: post #469
matsemann wrote: post #75

Earlier quoted context omitted.

> Meta AI isn't available yet in your country Where is it available? I got this in Norway.

This is so frustrating. Why don't they just make it available everywhere?

Because the EU requires them not to: https://ec.europa.eu/information_society/newsroom/image/docu...

Re: Meta Llama 3

#617

Post by greatpostman »

HarHarVeryFunny wrote: post #586
modeless wrote: post #314

Earlier quoted context omitted.

Yes. Llama 3 8B outperforms Llama 2 70B (in the instruct-tuned variants). "Chinchilla-optimal" is about choosing model size and/or dataset size to maximize the accuracy of your model under a fixed training budget (fixed number of floating point operations). For a given dataset size it will tell you the model size to use, and vice versa, again under the assumption of a fixed training budget. However, what people have…

What's interesting about the minimization of combined training + (model lifetime) inference cost is that that is going to look different for different companies, depending on what their inference volume is... Meta have a massive user base, and if they are using these models to run their own business, then that implies massive inference volume, and that it might make economic sense for them to put more money into trai…

Huge resources are being spent on these models at meta. Some very interesting software will come out of there in the next decade

Re: Meta Llama 3

#618

Post by cjbprime »

lolinder wrote: post #382
cjbprime wrote: post #262

Earlier quoted context omitted.

In general you can swap B for GB (and use the q8 quantization), so 8GB VRAM can probably just about work.

If you want to not quantize at all, you need to double it for fp16—16GB.

Yes, but I think it's standard to do inference at q8, not fp16.

Re: Meta Llama 3

#619

Post by cjbprime »

wongarsu wrote: post #551
worldsayshi wrote: post #252

Earlier quoted context omitted.

What is the reason for settling on 7/8 experts for mixture of experts? Has there been any serious evaluation of what would be a good MoE split?

A 19" server chassis is wide enough for 8 vertically mounted GPUs next to each other, with just enough space left for the power supplies. Consequently 8 GPUs is a common and cost efficient configuration in servers. Everyone seems to put each expert on a different GPU in training and inference, so that's how you get to 8 experts, or 7 if you want to put the router on its own GPU too. You could also do multiples of 8.…

(For a model of GPT-4's size, it could also be 8 nodes with several GPUs each, each node comprising a single expert.)

Re: Meta Llama 3

#620

Post by flakiness »

dhon_ wrote: post #614
selcuka wrote: post #594

Earlier quoted context omitted.

I'm seeing the same behaviour. It's as if they have a post-processor that evaluates the quality of the response after a certain number of tokens have been generated, and reverts the response if it's below a threshold.

I've noticed Gemini exhibiting similar behaviour. It will start to answer, for example, a programming question - only to delete the answer and replace it with something along the lines of "I'm only a language model, I don't know how to do that"

The linked article mentions these safeguards as the post-processing step.
Post reply on HN