I truly dislike them (and Mistral and others) purposely not comparing it against GPT3.5 or 4, of which a vast number of developers are familiar with.
It makes me thing it's nowhere near in terms of performance.
Meta Llama 3
451–460 of 965 posts
Re: Meta Llama 3
#452If any one is interesting in seeing how 400B model compares with other opensource models, here is a useful chart: https://x.com/natolambert/status/1780993655274414123
Fun fact, it's impossible to 100% the MMLU because 2-3% of it has wrong answers.
You just need to give the wrong answer ;)
Re: Meta Llama 3
#453Earlier quoted context omitted.
Good thing that he's only 39 years old and seems more energetic than ever to run his company. Having a passionate founder is, imo, a big advantage for Meta compared to other big tech companies.
Love how everyone is romanticizing his engineering mindset. But have we already forgotten that he was even more passionate about the metaverse which, as far as I can tell, was a 50B failure?
was a failure? they are still building it, when they shut down or sell off the division then you can call it a failure
Re: Meta Llama 3
#454I was curious how the numbers compare to GPT-4 in the paid ChatGPT Plus, since they don't compare directly themselves. Llama 3 8B Llama 3 70B GPT-4 MMLU 68.4 82.0 86.5 GPQA 34.2 39.5 49.1 MATH 30.0 50.4 72.2 HumanEval 62.2 81.7 87.6 DROP 58.4 79.7 85.4 Note that the free version of ChatGPT that most people use is based on GPT-3.5 which is much worse than GPT-4. I haven't found comprehensive eval numbers for the lates…
Via Microsoft Copilot (and perhaps Bing?) you can get access to GPT-4 for free.
Is Copilot free now?
Re: Meta Llama 3
#455Earlier quoted context omitted.
The world at large seems to hate Zuck but it’s good to hear from people familiar with software engineering and who understand just how significant his contributions to open source and raising salaries have been through Facebook and now Meta.
> his contributions to ... raising salaries It's fun to be able to retire early or whatever, but driving software engineer salaries out of reach of otherwise profitable, sustainable businesses is not a good thing. That just concentrates the industry in fewer hands and makes it more dependent on fickle cash sources (investors, market expansion) often disconnected from the actual software being produced by their teams.…
> but driving software engineer salaries out of reach of otherwise profitable, sustainable businesses is not a good thing.
I'm not convinced he's actually done that. Pretty much any 'profitable, sustainable business' can afford software developers.
Software developers are paid pretty decently, but (grabbing a couple of lists off of Google) it looks like there's 18 careers more lucrative than it (from a wage perspective), and computers-in-general are only 3 of the top 25 highest paying careers - https://money.usnews.com/careers/best-jobs/rankings/best-pay...
Medical, Legal, Finance, and Sales as careers (roughly in that order) all seem to pay more on average.
Re: Meta Llama 3
#456Quick thoughts - Major arch changes are not that major, mostly GQA and tokenizer improvements. Tokenizer improvement is a under-explored domain IMO. 15T tokens is a ton! 400B model performance looks great, can’t wait for that to be released. Might be time to invest in a Mac studio! OpenAI probably needs to release GPT-5 soon to convince people they are still staying ahead.
The money making opportunity is releasing PCs/laptops with 10x as much RAM.
Re: Meta Llama 3
#457Earlier quoted context omitted.
I dunno. I find a conviction in passion in founder CEOs that is missing in folks who replace them. Compare Larry & Sergey with Pichai, or Gates with Balmer.
Don't forget Gavin Belson and Action Jack Barker
Action Jack would still be at it but these days he prefers a nice piece of fish.
Re: Meta Llama 3
#458Earlier quoted context omitted.
Probably the EU laws are getting too draconian. I'm starting to see it a lot.
Claude has the same restriction [0], the whole of Europe (except Albania) is excluded. Somehow I don't think it is a retaliation against Europe for fining Meta and Google. I could be wrong, but a business decision seems more likely, like keeping usage down to a manageable level in an initial phase. Still, curious to understand why, should anyone here know more. [0] https://www.anthropic.com/claude-ai-locations
It's because of regulations!
The same reason that Threads was launched with a delay in EU. It simply takes a lot of work to comply with EU regulations, and by no surprise will we see these launches happen outside of EU first.
Re: Meta Llama 3
#459Earlier quoted context omitted.
Good thing that he's only 39 years old and seems more energetic than ever to run his company. Having a passionate founder is, imo, a big advantage for Meta compared to other big tech companies.
Love how everyone is romanticizing his engineering mindset. But have we already forgotten that he was even more passionate about the metaverse which, as far as I can tell, was a 50B failure?
Zuck's job is to have vision and take risks. He's doing that. He's going to encounter failures and I doubt he's still looking in the rearview mirror about it. And overall, Zuck has a tremendous amount of net success, to say the least.
Re: Meta Llama 3
#460Earlier quoted context omitted.
Love how everyone is romanticizing his engineering mindset. But have we already forgotten that he was even more passionate about the metaverse which, as far as I can tell, was a 50B failure?
The Quest is the top selling VR headset by a very large margin. He's well positioned to take that market when it eventually matures a bit. Once the tech gets there, say in a decade we might see most people primarily consume content via VR and phones. That's movies, games, TV, sporting events, concerts.
I just can’t imagine sitting with a headset on, next to my wife, watching the NFL. It could very well change for me, but it does not sound appealing.