The smallest model (7B) is supposed to outperform GPT-3.
Does anyone have any idea what hardware is needed to run this?
Facebook LLAMA is being openly distributed via torrents
441–450 of 719 posts
Re: Facebook LLAMA is being openly distributed via torrents
#442Earlier quoted context omitted.
It was already the most open language model in its class, given that the code for training and inference was available and it only used public data for training. For Google and OpenAIs offerings, have fun reimplementing it from descriptions in the paper (including small crucial details that they may have left out), training it for a month, and then wondering if the implementation or the training data is the reason yo…
The training code is not available.
Training code won't get you much if you don't have the infra/money to gather a suitable dataset or actually execute training. Plus if your goal is to "steal" or riff on the base model, it's already there in the weights. Also probably not difficult to figure out how to fine tune it once you have the weights and tokenizer.
Re: Facebook LLAMA is being openly distributed via torrents
#443Earlier quoted context omitted.
I'm curious if the blocking of adult content has to do with moralism, commercial interests, or something deeper. An eager to please conversational partner who can generate endless content seems quite dangerous and addictive, especially when it crosses over into romantic areas. There's already posts of people spending entire days interacting with LLMs, using as their therapist, romantic partner, etc. Combined with fin…
The "Joi" character from Blade Runner 2049 is an implementable reality, today. https://en.wikipedia.org/wiki/List_of_Blade_Runner_character...
creating the torment nexus one day at a time
Re: Facebook LLAMA is being openly distributed via torrents
#444Earlier quoted context omitted.
I don't think that's right - even if you had the full source code for either of those, it's extremely unlikely you'd be able to build them on your own machine.
Building them would be a challenge, but definitely not an insurmountable one. I’ve worked on a couple of C++ projects at a similar scale to Windows (millions of LOC) and the build systems were a major pain. But a determined engineer with the readme file and no other help could get it building in a week or so. (This probably says more about how hard it is to build C++ than anything else)
Fwiw C++ is hard to build specifically, but any large project with assets is a challenge to build.
Video games build systems are a thing onto themselves even taking the c++ issues out of the equation.
Hell even comparatively simole microservices in modern CRUD apps have resorted to docker to do away with pains of rebuilding stuff.
Re: Facebook LLAMA is being openly distributed via torrents
#445Re: Facebook LLAMA is being openly distributed via torrents
#446Earlier quoted context omitted.
That just shows their bias. If the code is complete, it's only a matter of time to figure out how to build it. May not take hours, but determined engineer should be able to figure it out.
I think you're being naive. Microsoft has spent 1000s of engineering years on their build system. You aren't going to just replicate that in a week.
As a user said below, you wouldn't need cross platform support, incremental builds, tests to run, etc.
Just getting the code to compile, link, baking in assets, etc. For a single architecture is a much more reasonable goal.
I'd imagine it'd be on the scale of 1-3months for an engineer to get working full time, but large error bars around this figure
Re: Facebook LLAMA is being openly distributed via torrents
#447Earlier quoted context omitted.
I'm curious if the blocking of adult content has to do with moralism, commercial interests, or something deeper. An eager to please conversational partner who can generate endless content seems quite dangerous and addictive, especially when it crosses over into romantic areas. There's already posts of people spending entire days interacting with LLMs, using as their therapist, romantic partner, etc. Combined with fin…
commercial interests. current organizations need recurring revenue or to sell shares at a higher price to investors this perpetual aspect is their achilles heel it is only a matter of time before an organization realizes they don't have to do a SaaS product to make a billion dollars. but for now, everyone's trying to make a hundred billion dollars and are steered into doing things that enthusiasts hate, so that they…
> it is only a matter of time before an organization realizes they don't have to do a SaaS product to make a billion dollars.
Most people recognize this. But venture backed startups (especially important for AI companies with high training costs) need to prove stickiness and reoccurring revenue to the investors. Conveniently a subscription proves both.
Subscribe and SaaS are just good for businesses (and tbh many purchasers of tech). I think it’s here to stay.
Re: Facebook LLAMA is being openly distributed via torrents
#448Earlier quoted context omitted.
> Electricity costs are basically irrelevant because the cards are so expensive. You mean in terms of money. I think this is exactly the problem that we have in CS, nobody really cares about CO2.
Pretty sure Facebook uses green energy for their datacenter, so the CO2 cost should be nothing.
I always feel there is an opportunity cost here though. If that green energy wasn’t being used for compute it could be available to heat someone’s home instead of them using dirty sources.
Re: Facebook LLAMA is being openly distributed via torrents
#449Earlier quoted context omitted.
No, I'm willing to bet the CO2 cost of the cards is also way higher than the electricity. Those things are built on the global supply chain, with materials potentially making multiple thousands of kms journeys between each step.
Long term I also imagine it's much cheaper to run these large model trainings on renewables. It's a very centralized process that doesn't necessarily need 100% availability. The manufacturing process, however, is totally decentralized, and NVIDIA mostly manufactures in China where coal is cheap.
A100s are manufactured in Taiwan.
Re: Facebook LLAMA is being openly distributed via torrents
#450Maybe this is an intentional leak to damage OpenAI. A supposedly better model by some accounts that strikes right at the heart of their business plan of selling access for $250k/year. One month of access to their service could buy a machine capable of running this leaked model. Facebook nerfs a potential upstart competitor to keep current big-tech cartel stable. Maybe this is a bit conspiratorial, but we live in the…
Not a conspiracy at all. See also IE, Android, Kubernetes...
Any links about IE, Android and Kubernetes? I'm not up on these being ops.