691–700 of 936 posts
Earlier quoted context omitted.
Of course it could be practical - provide the data. The fact of that society is a dystopian nightmare controlled by a few megacorporations that don't want free information does not justify outright changing the meaning of the language.
> provide the data Who? It's not their data.
Earlier quoted context omitted.
"Open weights" means you can use the weights for free (as in beer). "Open source" means you get the training dataset and the methodology. ~Nobody does open source LLMs.
If weights are not the source, then if they gave you the training data and scripts but not the weights, would that be "open source"?
Has anyone tried that?
Earlier quoted context omitted.
It means nothing because LLMs aren't software.
Do they not run on a computer?
Videos aren't software and neither are llms.
Earlier quoted context omitted.
Training code is only useful to people in academia, and the closest thing to "code you can modify" are open weights. People are framing this as if it was an open-source hierarchy, with "actual" open-source requiring all training code to be shared. This is not obvious to me, as I'm not asking people that share open-source libraries to also share the tools they used to develop them. I'm also not asking them to share al…
I dunno — if an open source project required, say, a proprietary compiler, that would diminish its open source-ness. But I agree it's not totally comparable, since the weights are not particularly analogous to machine code. We probably need a new term. Open Weights.
Earlier quoted context omitted.
The open source movement , from which the name derives, was about the freedom to make bespoke alterations to the software you choose to run. Provided you have reasonably widespread proficiency in industry standard tools, you can take something that's open source, modify that source, and rebuild/redeploy/reinterpret/re-whatever to make it behave the way that you want or need it to behave. This is in contrast to a comp…
>The open source movement, from which the name derives, was about the freedom to make bespoke alterations to the software you choose to run. The open source movement was a cash grab to make the free software movement more palatable to big corp by moving away from copy left licenses. The MIT license is perfectly open source and means that you can buy software without ever seeing its code.
Earlier quoted context omitted.
Meta does a good thing HN spends a day figuring out how it’s actually bad
No one said it was bad. It's just self interested (as companies generally are) and are using that to have a PR spin on the topic. But again, this is what all companies do and nothing about it is bad per se.
Earlier quoted context omitted.
>Nobody does open source LLMs. There are a bunch of independent, fully open source foundation models from companies that share everything (including all data). AMBER and MAP-NEO for example. But we have yet to see one in the 100B+ parameter category.
Sorry, the tilde before "nobody" is my notation for "basically nobody" or "almost nobody". I thought it was more common.
“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…
Sure, academia could build LLMs, and there is at least one large-scale project for that: https://gpt-nl.com/ On the other hand, this kind of models still need to demonstrate specific scientific value that goes beyond using a chatbot for generating ideas and summarizing documents.
So I fully agree that the research budget cuts in the past decades have been catastrophic, and probably have contributed to all the disasters the world is currently facing. But I think that funding prestigious super-projects is not the best way to spend funds.