Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Llama 3

llama.meta.com

Post reply on HN
Print view View original

141–150 of 965 posts

Re: Meta Llama 3

#141

Post by refulgentis »

oersted wrote: post #116
gliched_robot wrote: post #105

Earlier quoted context omitted.

Wild considering, GPT-4 is 1.8T.

Where did you find this number? Not doubting it, just want to get a better idea of how precise the estimate may be.

It's a really funny story that I comment about at least once a week because it drives me nuts.

1. After ChatGPT release, Twitter spam from influencers about chatGPT is one billion and GPT-4 is 1 trillion.

2. Semianalysis publishes a blog post claiming 1.8T sourced from insiders.

3. The way info diffusion works these days, everyone heard from someone else other than Semianalysis.

4. Up until about a month ago, you could confidently say "hey its just that one blog post" and work through it with people to trace their initial hearing of it back to the post.

5. nVidia press conference some time in the last month used the rumors as an example with "apparently" attached, and now people will tell you NVidia confirmed 1.8 trillion.

my $0.02: I'd bet my life GPT-4 isn't 1.8T, and I very much doubt its over 1 trillion. Like, lightning striking the same person 3 times in the same week.

Re: Meta Llama 3

#142

Post by typpo »

Public benchmarks are broadly indicative, but devs really should run custom benchmarks on their own use cases.

Replicate created a Llama 3 API [0] very quickly. This can be used to run simple benchmarks with promptfoo [1] comparing Llama 3 vs Mixtral, GPT, Claude, and others:

  prompts:
    - 'Answer this programming question concisely: {{ask}}'

  providers:
    - replicate:meta/meta-llama-3-8b-instruct
    - replicate:meta/meta-llama-3-70b-instruct
    - replicate:mistralai/mixtral-8x7b-instruct-v0.1
    - openai:chat:gpt-4-turbo
    - anthropic:messages:claude-3-opus-20240229

  tests:
    - vars:
        ask: Return the nth element of the Fibonacci sequence
    - vars:
        ask: Write pong in HTML
    # ...
Still testing things but Llama 3 8b is looking pretty good for my set of random programming qs at least.

Edit: ollama now supports Llama 3 8b, making it easy to run this eval locally.

  providers:
    - ollama:chat:llama3
[0] https://replicate.com/blog/run-llama-3-with-an-api

[1] https://github.com/typpo/promptfoo

Re: Meta Llama 3

#143

Post by janalsncm »

mfiguiere wrote: post #111

The instant generation of pictures as you type in meta.ai is really impressive!

It is. But I noticed something weird. If your prompt is “A cartoon of XYZ” and press enter the preview will be a cartoon but the other images will be weird realistic ones.

Re: Meta Llama 3

#144

Post by stareatgoats »

flemhans wrote: post #99
niek_pas wrote: post #82

Earlier quoted context omitted.

Got the same in the Netherlands.

Probably the EU laws are getting too draconian. I'm starting to see it a lot.

Claude has the same restriction [0], the whole of Europe (except Albania) is excluded. Somehow I don't think it is a retaliation against Europe for fining Meta and Google. I could be wrong, but a business decision seems more likely, like keeping usage down to a manageable level in an initial phase. Still, curious to understand why, should anyone here know more.

[0] https://www.anthropic.com/claude-ai-locations

Re: Meta Llama 3

#145

Post by janalsncm »

bg24 wrote: post #57

Last week, I made the explicit decision to circle through paid subscriptions of various providers every few months, instead of sticking to just ChatGPT. Glad to see Meta.ai on the list, in addition to Claude, Gemini, Mistral and Perplexity.

Any thoughts on which one you prefer? I have the ChatGPT subscription but curious about alternatives.

Re: Meta Llama 3

#146

Post by 12345hn6789 »

hermesheet wrote: post #36

Lots of great details in the blog: https://ai.meta.com/blog/meta-llama-3/ Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!

It's not open source or decentralized.

Re: Meta Llama 3

#147

Post by blackeyeblitzar »

J_cst wrote: post #98
robertlagrant wrote: post #80

Earlier quoted context omitted.

What is "source" regarding an LLM? Public training data and initial parameters?

Not an expert, but often weights are mentioned as not being open sourced. Happy to get corrected, as I'm not really sure.

Weights aren’t source because the goal of having open source software is that you can know how the software you’re consuming works, and you can produce the final software (the executable) using the source yourself. When you only have weights, you are getting something like the executable. Sure you can tweak it, but you don’t have the things you need to reproduce it or to examine how it works and validate it for your purposes. As such open weights are not in the spirit of open source.

Re: Meta Llama 3

#148

Post by jamesgpearce »

bbig wrote: post #4

They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.

That realtime `/imagine` prompt seems pretty great.

Re: Meta Llama 3

#149

Post by causal »

vacuumcl wrote: post #69
hermesheet wrote: post #36

Lots of great details in the blog: https://ai.meta.com/blog/meta-llama-3/ Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!

Comparing to the numbers here https://www.anthropic.com/news/claude-3-family the ones of Llama 400B seem slightly lower, but of course it's just a checkpoint that they benchmarked and they are still training further.

Indeed. But if GPT-4 is actually 1.76T as rumored, an open-weight 400B is quite the achievement even if it's only just competitive.
Post reply on HN