491–500 of 965 posts
Earlier quoted context omitted.
It is 100% possible for performance regressions to occur by changing the model pipeline and not the model itself. A system prompt is a part of said pipeline. Prompt engineering is surprisingly fragile.
Is that surprising? Seemed like a giant hack to me. Prompt engineering sure sounds better than hack though.
https://huggingface.co/models?sort=trending&search=llama-3
Sounds like quite the duplication of the model conversion. Few months ago it was TheBloke that did reliable model uploads. Nowadays, it's more straightforward to use https://ollama.com/library/llama3 as source to save time on searching through HF.
Earlier quoted context omitted.
Very interesting part around 5 mins in where Zuck says that they bought a shit ton of H100 GPUs a few years ago to build the recommendation engine for Reels to compete with TikTok (2x what they needed at the time, just to be safe), and now they are accidentally one of the very few companies out there with enough GPU capacity to train LLMs at this scale.
TikTok (ByteDance) is now building an AGI team to train and advance LLMs (towards AGI), probably after realizing they are in a similar scenario.
Earlier quoted context omitted.
Mistral is not “open source” either since we cannot reproduce it (the training data is not published). Both are open weight models, and they are both released under a license whose legal basis is unclear: it's not actually clear if they own any intellectual property over the model at all. Of course they claim such IP, but no court has ruled on this yet AFAIK and legislators could also enact laws that make these publi…
I have a hard time about the "cannot reproduce" categorization. There are places (e.g. in the Linux kernel? AMD drivers?) where lots of generated code is pushed and (apart from the rants of huge unwieldy commits and complaints that it would be better engineering-wise to get their hands on the code generator, it seems no one is saying the AMD drivers aren't GPL compliant or OSI-compliant? There are probably lots of OS…
Earlier quoted context omitted.
You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.
Also, being open source adds phenomenal value for Meta: 1. It attracts the world's best academic talent, who deeply want their work shared. AI experts can join any company, so ones which commit to open AI have a huge advantage. 2. Having armies of SWEs contributing millions of free labor hours to test/fix/improve/expand your stuff is incredible. 3. The industry standardizes around their tech, driving down costs and d…
Earlier quoted context omitted.
The Quest is the top selling VR headset by a very large margin. He's well positioned to take that market when it eventually matures a bit. Once the tech gets there, say in a decade we might see most people primarily consume content via VR and phones. That's movies, games, TV, sporting events, concerts.
I just can’t imagine sitting with a headset on, next to my wife, watching the NFL. It could very well change for me, but it does not sound appealing.
Similarly, I could imagine sitting next to my daughter, who is 2,500 miles away at college, watching the name together on a virtual screen we both share. And then playing mini-golf or table tennis together.
Different tools are appropriate for different use cases. Don't dismiss a hammer because it's not good at driving screws.
I just want to express how grateful I am that Zuck and Yann and the rest of the Meta team have adopted an open approach and are sharing the model weights, the tokenizer, information about the training data, etc. They, more than anyone else, are responsible for the explosion of open research and improvement that has happened with things like llama.cpp that now allow you to run quite decent models locally on consumer h…
Earlier quoted context omitted.
The Quest is the top selling VR headset by a very large margin. He's well positioned to take that market when it eventually matures a bit. Once the tech gets there, say in a decade we might see most people primarily consume content via VR and phones. That's movies, games, TV, sporting events, concerts.
I just can’t imagine sitting with a headset on, next to my wife, watching the NFL. It could very well change for me, but it does not sound appealing.
I just want to express how grateful I am that Zuck and Yann and the rest of the Meta team have adopted an open approach and are sharing the model weights, the tokenizer, information about the training data, etc. They, more than anyone else, are responsible for the explosion of open research and improvement that has happened with things like llama.cpp that now allow you to run quite decent models locally on consumer h…
You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.
He really is the last man standing from the web 2.0 days. I would have never believed I'd say this 10 years ago, but we're really fortunate for it. The launch of Quest 3 last fall was such a breath of fresh air. To see a CEO actually legitimately excited about something, standing on stage and physically showing it off was like something out of a bygone era.
They've got a console for it as well, https://www.meta.ai/ And announcing a lot of integration across the Meta product suite, https://about.fb.com/news/2024/04/meta-ai-assistant-built-wi... Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.