The future of foundation models is closed-source
blog.johnluttig.com
The future of foundation models is closed-source
1–10 of 14 posts
Re: The future of foundation models is closed-source
#2Considering this statement from the author, buried at the very end:
"I admit I have an allergic reaction when many open-source advocates expose their socialist tendencies from Europe, academia, or both. ... America's tech success is subject to endless criticism from those who missed out, but we handily won the last tech wave because American capitalism aligns users and companies for the long term."
I'd take everything else in this article with a huge grain of salt.
Re: The future of foundation models is closed-source
#3It’s closed source in one company unless every other company does something about it.
Re: The future of foundation models is closed-source
#4nice try but this is clumsy and one-dimensional in the parts that matter.. proof of unsuitability for purpose in this essay is that Mark F'in Zuckerberg is quoted directly as if that means anything.. (the guy is a liar). Secondly, LLMs are not the only AI application, they are the application that Joe Ordinary is going to get stuck talking to about your insurance premium; thirdly China is at this time arguably better at many AI tasks than anyone.. so a thin claim that China wants to copy something for free and nothing else, shows lack of sophistication.
The race is on behind the scenes, and the dollar valuations are proof of it. This essay starts a worthwhile conversation about the nature of the reproducible builds beast, but has little to bring in the way of insights or revelations IMHO
Re: The future of foundation models is closed-source
#5Considering this statement from the author, buried at the very end: "I admit I have an allergic reaction when many open-source advocates expose their socialist tendencies from Europe, academia, or both. ... America's tech success is subject to endless criticism from those who missed out, but we handily won the last tech wave because American capitalism aligns users and companies for the long term." I'd take everythin…
Wow seriously. It’s funny to hear “socialist tendencies from Europe” after having just listened to Yann LeCunn talk about his views on the need for open source AI. He says simply that the world needs diverse views and we can’t have a small number of companies from the west coast of the US serving as gatekeepers to the world’s knowledge. He’s from France and says the French government wouldn’t be satisfied with that arrangement either. He says we need freedom of information access for the same reason we need a free press.
So yeah LeCunn doesn’t want a small number of private companies to control our information. I wonder if the author views that as “socialist”.
Re: The future of foundation models is closed-source
#6Considering this statement from the author, buried at the very end: "I admit I have an allergic reaction when many open-source advocates expose their socialist tendencies from Europe, academia, or both. ... America's tech success is subject to endless criticism from those who missed out, but we handily won the last tech wave because American capitalism aligns users and companies for the long term." I'd take everythin…
This line in the second paragraph killed the author's credibility/objectivity for me; I didn't even get to the bit you reference:
> an unusual open-source alliance has formed among developers who want handouts
Handouts? Really? I've been an open source user and developer for decades now, and... "handouts"? I'm not even sure what to say to that.
At this point in my life I'm convinced that the style of capitalism practiced in the US is a failure. It gives you a lot of initial gains (which are, no doubt about it, quite important) but ultimately creates an untenable, inhumane situation for lots of people. I'm not convinced that European-style socialism (whatever that is; it's not like all the European countries are united on that front) is the answer either, though. I think a unique blend of capitalism and socialism is likely the only way forward without further wealth concentration and income inequality (not to mention planetary-scale ecological destruction, since the current forms of capitalism don't incentivize us to pursue sustainability in the long term). I'm not sure what that blend is, and I'm increasingly pessimistic that the people in power will allow us to get there, even if the alternatives will be disastrous for them as well.
Re: The future of foundation models is closed-source
#7> Open-source software started as an act of charity – the world owes the likes of Linus Torvalds and Fabrice Bellard for endowing humanity with Linux, Git, and FFmpeg.
So Linux is a simple act of charity from one person. If the author so fundamentally misunderstands what Open Source software is in principle and Linux is in particular then I struggle to see how he's qualified to come to such a definitive conclusion on a subject as complex as foundation models.
Re: The future of foundation models is closed-source
#8Considering this statement from the author, buried at the very end: "I admit I have an allergic reaction when many open-source advocates expose their socialist tendencies from Europe, academia, or both. ... America's tech success is subject to endless criticism from those who missed out, but we handily won the last tech wave because American capitalism aligns users and companies for the long term." I'd take everythin…
Wow seriously. It’s funny to hear “socialist tendencies from Europe” after having just listened to Yann LeCunn talk about his views on the need for open source AI. He says simply that the world needs diverse views and we can’t have a small number of companies from the west coast of the US serving as gatekeepers to the world’s knowledge. He’s from France and says the French government wouldn’t be satisfied with that a…
> we can’t have a small number of companies from the west coast of the US serving as gatekeepers to the world’s knowledge
I've lived on the west coast of the US for 20 years now, and the prospect of this terrifies me. In a "humanity's future is bleak" sort of way.
Re: The future of foundation models is closed-source
#9Considering this statement from the author, buried at the very end: "I admit I have an allergic reaction when many open-source advocates expose their socialist tendencies from Europe, academia, or both. ... America's tech success is subject to endless criticism from those who missed out, but we handily won the last tech wave because American capitalism aligns users and companies for the long term." I'd take everythin…
This whole article is a long-winded way of saying, "Hey, nerds! The gravy train's over! GPUs aren't free and Bill Gates was right! You can't eat freedom[0]! Either you point a gun at your users' temples or you starve to death!"
In some sense this is right, at least as long as all the research we're replicating boils down to "hey if you double the GPUs and data you get double the model quality". But whenever you make the "shit costs money" argument, you also entertain a moralistic co-argument: "I am entitled to supracompetitive profit". And this guy is definitely on the side of "pay up, capitalism works, you pinko commies".
The problem with this argument is that the datasets we currently train on are already stolen data. With the exception of extant FOSS licensing, which probably allow training[1], of course. The argument for closed models boils down to "we are entitled to be paid for our model but the artists and writers we stole[2] from are not".
[0] The clarion call of any savvy fascist
[1] Good luck figuring out how to comply with CC-BY-SA though
[2] In the Valentian sense where recording a movie onto a VHS tape makes you a home-invading rapist.
Re: The future of foundation models is closed-source
#10> Open-source software started as an act of charity – the world owes the likes of Linus Torvalds and Fabrice Bellard for endowing humanity with Linux, Git, and FFmpeg. So Linux is a simple act of charity from one person. If the author so fundamentally misunderstands what Open Source software is in principle and Linux is in particular then I struggle to see how he's qualified to come to such a definitive conclusion on…
Not just that, the author explicitly acknowledges a bias against open source, likening it to socialism (which the author clearly doesn't like) and welfare ("handouts", in the author's words).
I don't find their opinions worth considering all too deeply, given that bias.
If closed-source consolidation of models happens, it'll be because capitalism encourages and incentivizes centralization and secrecy, not because of any other reasons the author outlines.