251–260 of 936 posts
> Developers can run inference on Llama 3.1 405B on their own infra at roughly 50% the cost of using closed models like GPT-4o Does anyone have details on exactly what this means or where/how this metric gets derived?
Earlier quoted context omitted.
Llama’s code is open source: https://github.com/meta-llama/llama3/blob/main/llama/model.p...
This is like saying any python program is open source because the python runtime is open source. Inference code is the runtime; the code that runs the model. Not the model itself.
You can use that model with open data to train it from scratch yourself. Or you can load Meta’s open weights and have a working LLM.
The whole thing is interesting, but this part strikes me as potentially anticompetitive reasoning. I wonder what the lines are that they have to avoid crossing here?
Hard disagree. So far every big important model is closed-source. Grok is sort-of the only exception, and it's not even that big compared to the (already old) GPT-4. I don't see open source being able to compete with the cutting-edge proprietary models. There's just not enough money. GPT-5 will take an estimated $1.2 billion to train. MS and OpenAI are already talking about building a $100 billion training data cente…
Earlier quoted context omitted.
I actually think this is one of the rare times where the small guys interests are aligned with Meta. Meta is scared of a world where they are locked out of LLM platforms, one where OpenAI gets to dictate rules around their use of the platform much like Apple and Google dictates rules around advertiser data and monetization on their mobile platforms. Small developers should be scared of a world where the only competit…
Sure, but don't you think the "not getting locked out" is just the pre-requisite for their eventual goal of locking everyone else out?
I have faith they will continue to do what's in their best interests and if their best interests happen to align with mine, then I will support that. Just like how I don't bother killing the spider in my basement because it helps clean up the other bugs.
*I reserve the right to remove this praise if they abuse this open source model position in the future.
“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…
If anything, allocate compute to citizens.
It's a real shame that we're still calling Llama "open source" when at best it's "open weights." Not that anyone would go buy 100,000 H100s to train their own Llama, but words matter. Definitions matter.
Source versus weights seems like a really pedantic distinction to make. As you say, the training code and training data would be worthless to anyone who doesn't have compute on the level that Meta does. Arguably, the weights are source code interpreted by an inference engine, and realistically it's the weights that someone is going to want to modify through fine-tuning, not the original training code and data. The fa…
The acceptable use policy is seems fine. Don't use it to break the law, solicit sex, kill people, or lie.
Earlier quoted context omitted.
I don't find this argument super convincing. There's a pretty clear difference between the 'finetuning' offered via API by GPT4 and the ability to do whatever sort of finetuning you want and get the weights at the end that you can do with open weights models. "Brute forcing" is not the correct language to use for describing fine-tuning. It is not as if you are trying weights randomly and seeing which ones work on you…
"There's a pretty clear difference between the 'finetuning' offered via API by GPT4 and the ability to do whatever sort of finetuning you want and get the weights at the end that you can do with open weights models." Yes, the difference is that one is provided over a remote API, and the provider of the API can restrict how you interact with it, while the other is performed directly by the user. One is a SaaS solution…
You can take the weights and train LoRAs (which is close to fine-tuning), but you can also build custom adapters on top (classification heads). You can mix models from different fine-tunes or perform model surgery (adding additional layers, attention heads, MoE).
You can perform model decomposition and amplify some of its characteristics. You can also train multi-modal adapters for the model. Prompt tuning requires weights as well.
I would even say that having the model is more potent in the hands of individual users than having the dataset.