Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s policies hinder reproducible research on language models

aisnakeoil.substack.com

Post reply on HN
Print view View original

181–190 of 394 posts

Re: OpenAI’s policies hinder reproducible research on language models

#181

Post by zarzavat »

sva_ wrote: post #22

Since OpenAI didn't release the parameter count of GPT-4, I've been wondering/doubting if it is really much bigger than GPT-3. The release of GPT-3.5 has shown that they've found ways of drastically cutting down compute costs (an order of magnitude) while maintaining or even improving the quality of the model's outputs. Perhaps the reason that they didn't release the specifics of GPT-4 might be in part due to them wa…

ChatGPT-4 is definitely slower than GPT-3.5 (and way slower than 3.5-turbo). What could be the reason for that other than much larger parameter count?

I agree that the capabilities seem overhyped. In my subjective experience, 4 seems a little better than 3.5 but not by a huge amount. We just have OpenAI’s cherry-picked word that it‘s this incredible advance.

Re: OpenAI’s policies hinder reproducible research on language models

#182

Post by version_five »

Some of the blame should rest with researchers, and referees of their work. I agree with the authors here, but I also think it's a poor choice to base your research on a closed model, and for reviewers not to accept research that has a dependency like this. How did it become standard academic practice to work with something like this that you cannot interrogate.

Re: OpenAI’s policies hinder reproducible research on language models

#183

Post by villgax »

On a side note, if they scraped & built a portion of their corpus then it is fair to use their outputs to do whatever we want with their outputs. Should have not provided a free tier if they were so concerned, like what did they expect people would use an LLM like that for lol.

Re: OpenAI’s policies hinder reproducible research on language models

#184

Post by bitL »

mach1ne wrote: post #118
ftxbro wrote: post #66

Historically, researchers at some of the biggest tech companies had permission to publish their results. Presumably it was mutually beneficial; many researchers held dual positions in academia and industry, and publishing cool models could attract good researchers to the company. But stuff got real. They discovered a path to super-human cognition that scales directly with money and computer chips. Now these companies…

Super-human cognition? Hard to say. GPT-4 does raise the possibility of a machine writing smarter text than a human. What perplexes me is that since GPT is a predictor, it shouldn’t be able to write the smartest text - it should write the average text (since that has the largest frequency in the training set). Yet this does not seem to be the case. Is it inevitable that despite the quality of the data, better models…

I think you misunderstood how the generation of text works. For each new token it samples probabilities given previous tokens, not averages, then chooses some token from the top k as the next one with rules that penalize repetition of some order.

Moreover, there is no upper bound for transformers found yet, i.e. the larger the model is and the more data is used for training, the better it performs. It's literally about who is able to throw more money at it at this point, with some closely guarded secrets like warm up steps, training schedules etc. There is also the overfitting effect where one pushes training far beyond overfitting (validation loss growing again) as with transformers at some point the overfitting stops, validation loss starts dropping again and that's when the magic starts happening and money are burnt for scale.

Re: OpenAI’s policies hinder reproducible research on language models

#185

Post by withinboredom »

mach1ne wrote: post #118
ftxbro wrote: post #66

Historically, researchers at some of the biggest tech companies had permission to publish their results. Presumably it was mutually beneficial; many researchers held dual positions in academia and industry, and publishing cool models could attract good researchers to the company. But stuff got real. They discovered a path to super-human cognition that scales directly with money and computer chips. Now these companies…

Super-human cognition? Hard to say. GPT-4 does raise the possibility of a machine writing smarter text than a human. What perplexes me is that since GPT is a predictor, it shouldn’t be able to write the smartest text - it should write the average text (since that has the largest frequency in the training set). Yet this does not seem to be the case. Is it inevitable that despite the quality of the data, better models…

Clearly written “average” text will always be better than unclear “smart text”

Re: OpenAI’s policies hinder reproducible research on language models

#186

Post by danpalmer »

gwd wrote: post #178

If you came here after only reading the headline, you missed what the complaint is actually about: It's not that GPT-4 is closed source. It's that access to `codex` model was pulled with only three days notice, and the model itself was not open-sourced. Since apparently a large number of researchers were writing papers which used that particular model, that means all of those research papers are now non-reproducible.…

And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.

Re: OpenAI’s policies hinder reproducible research on language models

#187

Post by bitL »

Tenoke wrote: post #153
bitL wrote: post #45

Earlier quoted context omitted.

It's still much worse than 1M context on 16GB VRAM with Reformer, but at the cost of inference speed. And you can use FlashAttention in your own models to get a more efficient/sparse attention now as well.

The quality with reformer is much much worse, it's not really comparable.

Yeah, but it fits on a single GPU. Now imagine it scaled across 1000 GPUs.

Re: OpenAI’s policies hinder reproducible research on language models

#189

Post by gpderetta »

skissane wrote: post #156
AstixAndBelix wrote: post #147

Earlier quoted context omitted.

The problem is that AI research is moving incredibly fast. You might train a LLN today for $5M but a year from now the competition will have implemented an absolutely killer feature that needs $10M worth of training

AI research isn't particularly expensive. US$10 million to train a new model? Other fields have R&D budgets measured in the billions. I bet if you were a senior researcher at OpenAI, and you decided to quit and start a competing firm, there'd be a whole line of investors wanting to give you a lot more than US$10 million. And you don't need to be coming first in the technology race to make money. A lot of people would…

Agree completely. The state of the art is probably going to always be closed and proprietary, but, especially with hardware becoming more and more powerful, training a custom model is not going to be beyond the budget and capabilities of even small organizations.

Re: OpenAI’s policies hinder reproducible research on language models

#190

Post by gwd »

atleastoptimal wrote: post #32

I understand any individual's company anti-competitive measures. OpenAI looks at Google the same way Apple looked at IBM in the 80s. What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 before it was…

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. From everything I've heard and seen, the actual researchers at OpenAI are trying to take seriously the risk that a super-intelligent AI might destroy the human race. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people they work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reckless, helping set the human race on a path for certain doom.

From that perspective, the opening of ChatGPT has actually been very effective at raising awareness. All the way back in GPT-1 they were trying to raise warnings, but those warnings didn't get much popular traction. Now that so many people have used ChatGPT (or Bing), I'm now having conversations about what computers "know" and "want" with my aunt on Facebook.

Furthermore, if OpenAI has the best tools and sells them to everyone at a reasonable price, then there's a reduced incentive for other people to make their own tools. Whereas, if they were to close off access to the API, and only offer it to large corporations, there would be much more incentive for people to experiment with AI on their own -- and in doing so, possibly create an "un-aligned" super-intelligent AI which would destroy the human race.

So my prediction is that given their motivations, they will 1) stop releasing details of their models to anyone other than research organizations they consider careful enough 2) continue to sell reasonably-priced access to the APIs, to reduce the risk that other people will step up to fill the demand who are less careful.

Post reply on HN