Earlier quoted context omitted.
Via Microsoft Copilot (and perhaps Bing?) you can get access to GPT-4 for free.
Is Copilot free now?
There's a free tier and a 'pro' tier.
Meta Llama 3
661–670 of 965 posts
Re: Meta Llama 3
#662Earlier quoted context omitted.
> his contributions to ... raising salaries It's fun to be able to retire early or whatever, but driving software engineer salaries out of reach of otherwise profitable, sustainable businesses is not a good thing. That just concentrates the industry in fewer hands and makes it more dependent on fickle cash sources (investors, market expansion) often disconnected from the actual software being produced by their teams.…
> Nor is it great for the yet-to-mature craft that high salaries invited a very large pool of primarly-compensation-motivated people who end up diluting the ability for primarily-craft-motivated people to find and coordinate with each other in pursuit of higher quality work and more robust practices. It's great to enjoy programming, and to enjoy your job. But we live under capitalism. We can't fault people for just w…
Pushing salary lowers help the society at large, or at least that’s the thesis of OP. While it sucks for SWE, I actually kind of agree. The skyrocketing of SWE salary in the US, and the slow progress US is making towards normalizing/reducing it does not help US competitiveness. I would not fault Meta for this though, as much as US society at large.
SWE should enjoy it while they can before salary becomes similar to other engineering trades.
Re: Meta Llama 3
#663Earlier quoted context omitted.
When did you join Google?
mid-2000s just prior to the ipo.
Oh, I see, that must have been quite the journey.
I joined in 2014, and even I saw the changes in just a few years when I was there.
Still I was a bit baffled reading all the lamenters: I joined late enough that I had no illusions and always saw Google as doing pretty well for an 'enterprise', instead of feeling and expressing constant disappointment that the glory days were over.
Re: Meta Llama 3
#664Last week, I made the explicit decision to circle through paid subscriptions of various providers every few months, instead of sticking to just ChatGPT. Glad to see Meta.ai on the list, in addition to Claude, Gemini, Mistral and Perplexity.
Just use the openrouter API to switch models with a chat frontend like Chatbox for Mac
Re: Meta Llama 3
#665Earlier quoted context omitted.
It is 100% possible for performance regressions to occur by changing the model pipeline and not the model itself. A system prompt is a part of said pipeline. Prompt engineering is surprisingly fragile.
Absolutely! That was covered in the tweet link. If you're suggesting they're lying*, I'm happy to extract it and check. * I don't think you are! I've looked up to you a lot over last year on LLMs btw, just vagaries of online communication, can't tell if you're ignoring the tweet & introducing me to idea of system prompts, or you're suspicious it changed recently. (in which case, I would want to show off my ability to…
I was agreeing with the tweet and think Anthropic is being honest, my comment was more for posterity since not many people know the difference between models and pipelines.
Thanks for liking my work! :)
Re: Meta Llama 3
#666Earlier quoted context omitted.
You can see from Zuck's interviews that he is still an engineer at heart. Every other big tech company has lost that kind of leadership.
Also, being open source adds phenomenal value for Meta: 1. It attracts the world's best academic talent, who deeply want their work shared. AI experts can join any company, so ones which commit to open AI have a huge advantage. 2. Having armies of SWEs contributing millions of free labor hours to test/fix/improve/expand your stuff is incredible. 3. The industry standardizes around their tech, driving down costs and d…
[dead]
Re: Meta Llama 3
#667Earlier quoted context omitted.
Is “reproducibility” actually the right term here? It’s a bit like arguing that Linux is not open source because you don’t have every email Linus and the maintainers ever received. Or that you don’t know what lectures Linus attended or what books he’s read. The weights “are the thing” in the same sense that the “code is the thing”. You can modify open code and recompile it. You can similarly modify weights with fine…
> It’s a bit like arguing that Linux is not open source because you don’t have every email Linus and the maintainers ever received. Or that you don’t know what lectures Linus attended or what books he’s read. Linux is open source, because you can actually compile it yourself! You don't need Linus's email for that (and if you needed some secret cryptographic key on Linus' laptop to decrypt and compile the kernel, then…
> the same way you can edit the binary of a proprietary game to disable DRMs, that doesn't make it open-source either
This is where I have to disagree. Continuing the training of an open model is the same process as the original training run. It's not a fundamentally different operation.
Re: Meta Llama 3
#668Earlier quoted context omitted.
Where? I only see comparisons to Mistral 7B and Mistral Medium, which are totally different models.
https://ai.meta.com/blog/meta-llama-3/ has it about a third of the way down. It's a little bit better on every benchmark than Mixtral 8x22B (according to Meta).
Oh cool! But at the cost of twice the VRAM and only having 1/8th of the context, I suppose?
Re: Meta Llama 3
#669https://github.com/meta-llama/llama3/blob/main/LICENSE Llama is not open source. It's corporate freeware with some generous allowances. Open source licenses are a well defined thing. Meta marketing saying otherwise doesn't mean they get to usurp the meaning of a well understood and commonly used understanding of the term "open source." https://opensource.org/license Nothing about Meta's license is open source. It's a…
> " Nothing about Meta's license is open source. It's a carefully constructed legal agreement intended to prevent any meaningful encroachment by anyone, ever, into any potential Meta profit, and to disavow liability to prevent reputational harm in the case of someone using their freeware for something embarrassing. " You seem to be making claims that have little connection to the actual license. The license states yo…
any scale restrictions plus the "etc." means it's not open source.
Re: Meta Llama 3
#670I just want to express how grateful I am that Zuck and Yann and the rest of the Meta team have adopted an open approach and are sharing the model weights, the tokenizer, information about the training data, etc. They, more than anyone else, are responsible for the explosion of open research and improvement that has happened with things like llama.cpp that now allow you to run quite decent models locally on consumer h…
It's crazy how the managerial executive class seems to resent the vital essence of their own companies. Based on the behavior, nature, stated beliefs and interviews I've seen of most tech CEOs and CEOs in general, there seems to be almost a natural aversion to talking about things in non hyper-abstracted terms. I get the feeling that the nature of the corporate world is often better understood as a series of rituals…
It's because Elon, Mark, and Jensen are true founders. They aren't MBAs who got voted in because shareholders thought they would make them the most money in the shortest amount of time.