891–900 of 965 posts
Earlier quoted context omitted.
Llama is open weight, not open source. They don’t release all the things you need to reproduce their weights.
Has anyone tested how close you need to be to the weights for copyright purposes?
Earlier quoted context omitted.
I feel like Lex has gone full 'both sides' at this point, waiting for him to have Alex Jones on at this point. There is no real commentary to pull from his interviews, at best you get some interesting stories but not the truth.
That is a strength, not a weakness. It's valuable to see why people, even those with whom we disagree, think the way they do. There's already far too much of a tendency to expel heretics in today's society, so the fact that Lex just patiently listens to people is a breath of fresh air.
Earlier quoted context omitted.
They also stated that they are still training larger variants that will be more competitive: > Our largest models are over 400B parameters and, while these models are still training, our team is excited about how they’re trending. Over the coming months, we’ll release multiple models with new capabilities including multimodality, the ability to converse in multiple languages, a much longer context window, and stronge…
Anyone have any informed guesstimations as to where we might expect a 400b parameter model for llama 3 to land benchmark wise and performance wise, relative to this current llama 3 and relative to GPT-4? I understand that parameters mean different things for different models, and llama two had 70 b parameters, so I'm wondering if anyone can contribute some guesstimation as to what might be expected with the larger mo…
Earlier quoted context omitted.
His engineering mindset made him blind to the fact the metaverse was a product that nobody wanted or needed. In one of the Fridman interviews, he goes on and on about all the cool technical challenges involved in making the metaverse work. But when Fridman asked him what he likes to do in his spare time, it was all things that you could precisely not do in the metaverse. It was baffling to me that he failed to connec…
Yes, I thought the same exact thing. Seemed so odd to hear him gush over his foiling and MMA while simultaneously expecting everyone else to migrate to the metaverse.
Maybe I am an outlier, but when in a conversation about work-related things someone asks “what do you like to do in your free time”, I believe the implication here is that there is a silent “…to do in your free time [outside of work]”.
Answering that question with more stuff related to work project typically falls somewhere on the spectrum between pandering to the audience and cringe.
No idea how this concept can even count as novel on HN, where a major chunk of users that are software devs keep talking about hobbies like woodworking/camping/etc. (aka hobbies that are typically as far removed from the digital realm as possible).
Imo Zuck talking about MMA being his personal free time hobby is about as odd as a software dev talking about being into woodworking. In other words, not at all.
Earlier quoted context omitted.
> Nor is it great for the yet-to-mature craft that high salaries invited a very large pool of primarly-compensation-motivated people who end up diluting the ability for primarily-craft-motivated people to find and coordinate with each other in pursuit of higher quality work and more robust practices. It's great to enjoy programming, and to enjoy your job. But we live under capitalism. We can't fault people for just w…
Pushing salary lowers help the society at large, or at least that’s the thesis of OP. While it sucks for SWE, I actually kind of agree. The skyrocketing of SWE salary in the US, and the slow progress US is making towards normalizing/reducing it does not help US competitiveness. I would not fault Meta for this though, as much as US society at large. SWE should enjoy it while they can before salary becomes similar to o…
Earlier quoted context omitted.
> but driving software engineer salaries out of reach of otherwise profitable, sustainable businesses is not a good thing. I'm not convinced he's actually done that. Pretty much any 'profitable, sustainable business' can afford software developers. Software developers are paid pretty decently, but (grabbing a couple of lists off of Google) it looks like there's 18 careers more lucrative than it (from a wage perspecti…
Few viable technology businesses and non-technology busiesses with internal software departments were prepared to see their software engineers suddenly suddenly expect doctor or lawyer pay and can't effectively accomodate the change. They were largely left to rely on loyalty and other kinds of fragile non-monetary factors to preserve their existing talent and institutuonal knowledge and otherwise scavenge for scraps…
Earlier quoted context omitted.
It’s easy to populate your feed with things you specifically want to watch: watch the stuff you’re interested in and swipe on the things that don’t interest you.
Reels don't interest me, they are just showed in my face whenever I use Facebook (or should I say Face-butt?). It's impossible to hide without using a custom script/adblock, which I ended up doing, but the only long term, cross device solution is to simply to delete the Facebook account.
Earlier quoted context omitted.
Losers & Winners from Llama-3-400B Matching 'Claude 3 Opus' etc.. Losers: - Nvidia Stock : lid on GPU growth in the coming year or two as "Nation states" use Llama-3/Llama-4 instead spending $$$ on GPU for own models, same goes with big corporations. - OpenAI & Sam: hard to raise speculated $100 Billion, Given GPT-4/GPT-5 advances are visible now. - Google : diminished AI superiority posture Winners: - AMD, intel: th…
The memory chip companies were done for, once Bill Gates figured out no one would ever need more than 64K of memory
Earlier quoted context omitted.
It's a blob that costs over $10,000,000 in electricity costs to compile. Even if they released everything only the rich could push go.
In today's Dwarkesh interview, Zuckerberg talks about energy becoming a limit for future models before cost or access to hardware does. Apparently current largest datacenters consume about 100MW, but Zuck is considering future ones consuming 1GW which is the output of typical nuclear reactor! So, yeah, unless you own your own world-class datacenter, complete with the nuclear reactor necessary to power the training ru…
I think Zuck's discussion of energy being the limiting factor was one of the more interesting and surprising things to come out of the Dwarkesh interview. We're used to discussion of the $1B, $10B, $100B training runs becoming unsustainable, and chip shortages as an issue, but (to me at least!) it was interesting to see Zuck say that energy usage will be a disruptor before those do (partly because of lead times and regulations in expanding power supply, and bringing it in to new data centers). The sheer magnitude of projected power consumption needed is also interesting.
Earlier quoted context omitted.
In practice this isn't a matter of how you or I interpret this license - it's a matter of how watertight it is legally. There's no reason to suppose that terms of any commercial licensing agreement would be onerous. At this stage at least these models are all pretty fungible and could be swapped out without much effort, so Meta would be competing with other companies for your business, if they want it. If they don't…
> In any case, don't argue it with me No argument here. You can either read it or you can't. :)