IMO established companies (Meta, Google, etc) had their researchers publish papers as a competitive benefit or way to attract talent from academia (a researcher wouldn't want to stop publishing).
Companies didn't see an issue with doing that because those papers were not "giving away" the core of the company, for example, Facebook's DeepFace paper from 2014 couldn't hurt its ad business.
OpenAI on the other hand will probably be as closed as they can be with their LLMs.
OpenAI’s policies hinder reproducible research on language models
11–20 of 394 posts
Re: OpenAI’s policies hinder reproducible research on language models
#12All these research science bureaucrats at Big Tech could have released LLM models or tried to develop what OpenAI did. But none of them did. We should applaud OpenAI for the innovation and let them do as they please.
[deleted]
Re: OpenAI’s policies hinder reproducible research on language models
#13We need more AI skeptics like this to dismantle and cut through the hype and to unveil the limits of AI that the hype squad continues to push this narrative to pump their AI grift projects.
OpenAI is the ring-leader of this bait and switch using faux 'AI safety' excuses to close their research and models and even their papers for researchers. It is essentially a majority owned Microsoft® AI division.
Re: OpenAI’s policies hinder reproducible research on language models
#14I'm confused why people expect this stuff to be free? I'm surprised OpenAI was so open about their research so far. I don't blame them at all for not publishing the information. This stuff costs real money.
Re: OpenAI’s policies hinder reproducible research on language models
#15What is the incentive to build and maintain a product that matches the researchers specs?
Re: OpenAI’s policies hinder reproducible research on language models
#16I'm quite sure even OpenAI themselves aren't sure if they can reproduce the current models from the scratch. Unless the computing becomes much more powerful and much cheaper, LLM is more or less a rocket science (i.e. hella expensive trial and error). It's not easy to burn lots of dollars just to get what's already there.
We have to identify a better method. You can't trial and error a pivotal act.
Re: OpenAI’s policies hinder reproducible research on language models
#17I'm quite sure even OpenAI themselves aren't sure if they can reproduce the current models from the scratch. Unless the computing becomes much more powerful and much cheaper, LLM is more or less a rocket science (i.e. hella expensive trial and error). It's not easy to burn lots of dollars just to get what's already there.
I don't even care if it's reproducible or not. I care it gives me correct responses to my questions and that's all.
Re: OpenAI’s policies hinder reproducible research on language models
#18The article seems premised on a misunderstanding that OpenAI is a research lab. For all intents and purposes, it’s a for-profit subsidiary of Microsoft, and there’s little financial incentive for it to maintain old models for others’ benefit.
We're under no such misapprehension and we're keenly aware that this is an uphill battle. The issue is that LLMs have become part of the infrastructure of the Internet. Companies that build infrastructure have a responsibility to society, and we're documenting how OpenAI is reneging on that responsibility. Hindering research is especially problematic if you take them at their word that they're building AGI. If infrastructure companies don't do the right thing, they eventually get regulated (and if you think that will never happen, I have one word: AT&T).
Finally, even if you don't care about research at all, the article mentions OpenAI's policy that none of their models going forward will be stable for more than 3 months, and it's going to be interesting to use them in production if things are going to keep breaking regularly.
Re: OpenAI’s policies hinder reproducible research on language models
#19I'm confused why people expect this stuff to be free? I'm surprised OpenAI was so open about their research so far. I don't blame them at all for not publishing the information. This stuff costs real money.
It might be less confusing if you consider that OpenAI was originally a non-profit. That it was even possible for them to end up in this state has massively undermined any trust I have in non-profits as a steward.
https://www.vice.com/en/article/5d3naz/openai-is-now-everyth...
> OpenAI was founded in 2015 as a nonprofit research organization by Altman, Elon Musk, Peter Thiel, and LinkedIn cofounder Reid Hoffman, among other tech leaders. In its founding statement, the company declared its commitment to research “to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.” The blog stated that “since our research is free from financial obligations, we can better focus on a positive human impact,” and that all researchers would be encouraged to share "papers, blog posts, or code, and our patents (if any) will be shared with the world."
> By March 2019, OpenAI shed its non-profit status and set up a “capped profit” sector, in which the company could now receive investments and would provide investors with profit capped at 100 times their investment.
Re: OpenAI’s policies hinder reproducible research on language models
#20I'm quite sure even OpenAI themselves aren't sure if they can reproduce the current models from the scratch. Unless the computing becomes much more powerful and much cheaper, LLM is more or less a rocket science (i.e. hella expensive trial and error). It's not easy to burn lots of dollars just to get what's already there.
I don't even care if it's reproducible or not. I care it gives me correct responses to my questions and that's all.
There's really no way to be sure that it will.