Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s policies hinder reproducible research on language models

aisnakeoil.substack.com

Post reply on HN
Print view View original

221–230 of 394 posts

Re: OpenAI’s policies hinder reproducible research on language models

#221

Post by hcrean »

lionkor wrote: post #150
ugh123 wrote: post #140

Earlier quoted context omitted.

>performance on standardized tests? That doesn't necessarily seem like the best metric for what the LLM tries to be. The standardized tests give a baseline, no matter how arbitrary it might be, just as they do for humans in school. Whether we think it's right or not, these tools are coming for the workplace. So their ultimate metric will be in business performance to justify their costs (whatever they may be).

GPT 3.5 had trouble understanding when I told it "Say 2 bob are a beb, how many beb per bob are there?" and it wrote a goddamn essay about shoes. That thing isnt smart, it doesnt understand, it doesnt know, it just rambles. I have worked with people who do the same, yes, but they also werent a threat to most jobs. I said it before, and I will say it again: If ChatGPT 3,4,5,... can take your job, maybe youre not reall…

I think right here we have an example AI reproducibility problem. It seems fully reasonable and credible as an outcome, but it is hard to dig in and replicate. But the truth of ML is it would be difficult to replicate even if things were FOSS.

Re: OpenAI’s policies hinder reproducible research on language models

#222

Post by totalhack »

My hope is open research and open source collaboration will continue to lead to breakthroughs, most importantly lowering the barrier for entry to training such capable models.

It's still relatively early days for this technology; if model research and processing power developments find an order of magnitude or two efficiency gain over the next decade maybe OpenAI's closed approach will no longer matter. Maybe that's wishful thinking though.

Re: OpenAI’s policies hinder reproducible research on language models

#223

Post by sverona »

oefnak wrote: post #211
gwd wrote: post #203

Earlier quoted context omitted.

> There are always financial incentives. A useful question to ask yourself is, "How would I know if I were wrong? What kind of evidence would convince me that a decision was not driven primarily by financial incentives?" If your "model" is equally compatible with all possible observations -- if anything that happens actually confirms the model rather than disproving it -- then it's not actually that useful as a model…

You sound rational. Do you not agree with the possibility of AI doom soon?

It's possible, but the views of the AI alignment community so far as I can tell are being skewed way too far towards nihilistic doomerism by the influence of Yudkowsky, who apparently believes that we're all gonna die in a few years and there's nothing anyone can do to stop it. [0]

[0] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

Re: OpenAI’s policies hinder reproducible research on language models

#224

Post by mlwart »

KyeRussell wrote: post #188
MagicMoonlight wrote: post #177

This is why we need a lawsuit against them. They’ve harvested everyone’s data unlawfully to train their model and now they’re cutting off access to starve the competition.

[flagged]

What is the article about then? They cut off researchers to starve the research competition?

That's another interpretation, perhaps they cut off researchers and true open source competition and business competitors.

Re: OpenAI’s policies hinder reproducible research on language models

#225

Post by UncleEntity »

phkahler wrote: post #210
gwd wrote: post #178

If you came here after only reading the headline, you missed what the complaint is actually about: It's not that GPT-4 is closed source. It's that access to `codex` model was pulled with only three days notice, and the model itself was not open-sourced. Since apparently a large number of researchers were writing papers which used that particular model, that means all of those research papers are now non-reproducible.…

>> An obvious thing to do would be to either open-source older models (including the weights) when retiring them; or possibly transfer them to an institution who see their role specifically as serving as an archive Another obvious thing to do is do your research on non-commercial or open source things that can not be taken away from you. Sorry, I don't mean for the snark present in that statement. The frustration lie…

Exactly, expecting a company to maintain a project forever (or give it away) just because they were used in research projects is ludicrous.

Maybe they did the research when “open” in their name meant something but it has been obvious for quite a while that ship has sailed.

Re: OpenAI’s policies hinder reproducible research on language models

#226

Post by UncleEntity »

anonymousDan wrote: post #214
gwd wrote: post #190

Earlier quoted context omitted.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. From everything I've heard and seen, the actual researchers at OpenAI are trying to take seriously the risk that a super-intelligent AI might destroy the human race. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money…

What a bunch of BS. The only reason they are keeping it private is for commercial gain .

Just look at how much flak the stable diffusion folks got for the deep fake porn (and whatever else people were pearl grasping over) and tell me how a corporation will ever release a model.

Meta was a fluke but they also did due diligence and made it look like they tried to do a responsible release — right up until someone put it on BitTorrent.

Re: OpenAI’s policies hinder reproducible research on language models

#227

Post by Torifyme12 »

sverona wrote: post #223
oefnak wrote: post #211

Earlier quoted context omitted.

You sound rational. Do you not agree with the possibility of AI doom soon?

It's possible, but the views of the AI alignment community so far as I can tell are being skewed way too far towards nihilistic doomerism by the influence of Yudkowsky, who apparently believes that we're all gonna die in a few years and there's nothing anyone can do to stop it. [0] [0] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

Yudkowsky

Man how the fuck does that guy keep popping up in the most random places starting fights?

Re: OpenAI’s policies hinder reproducible research on language models

#228

Post by bheadmaster »

gwd wrote: post #203
Dalewyn wrote: post #199

Earlier quoted context omitted.

>Most of those individuals have absolutely nothing commercial to gain from stopping AI research. There are always financial incentives. Like it or not, there's a lot of money on the line in the "AI" industry; if someone wants that industry to go a certain way, they definitely have something to gain or lose financially. In particular, it's obvious to anyone who's been paying attention that the west halting/ceding AI r…

> There are always financial incentives. A useful question to ask yourself is, "How would I know if I were wrong? What kind of evidence would convince me that a decision was not driven primarily by financial incentives?" If your "model" is equally compatible with all possible observations -- if anything that happens actually confirms the model rather than disproving it -- then it's not actually that useful as a model…

Applying your own reasoning, what evidence would convince you that every money-making industry is necessarily driven by profit?

Re: OpenAI’s policies hinder reproducible research on language models

#229

Post by DoingIsLearning »

jillesvangurp wrote: post #196
atleastoptimal wrote: post #32

I understand any individual's company anti-competitive measures. OpenAI looks at Google the same way Apple looked at IBM in the 80s. What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 before it was…

You are assuming openai is going to end up with a monopoly on all this. IMHO the opposite is going to happen. There are going to be a multitude of companies and researchers competing on outdoing what they are doing in terms of quality, cost, and use cases. If big companies put a straight jacket in place to limit access, constrain usage, etc., that just creates the opportunity for others to step up and grab some marke…

Indeed, I just saw a demo of Adobe Firefly, and the surprising thing to me is the whole thing was developed internally from data they control.

Looking at Nvidia's rental solutions for Nvidia’s A100, it really feels like Future products will be driven by who is sitting on the biggest closed source training datasets more so than this specific success from OpenAI's research.

Re: OpenAI’s policies hinder reproducible research on language models

#230

Post by circuit10 »

bitL wrote: post #184
mach1ne wrote: post #118

Earlier quoted context omitted.

Super-human cognition? Hard to say. GPT-4 does raise the possibility of a machine writing smarter text than a human. What perplexes me is that since GPT is a predictor, it shouldn’t be able to write the smartest text - it should write the average text (since that has the largest frequency in the training set). Yet this does not seem to be the case. Is it inevitable that despite the quality of the data, better models…

I think you misunderstood how the generation of text works. For each new token it samples probabilities given previous tokens, not averages, then chooses some token from the top k as the next one with rules that penalize repetition of some order. Moreover, there is no upper bound for transformers found yet, i.e. the larger the model is and the more data is used for training, the better it performs. It's literally abo…

You missed the point they were making, which is that the probabilities it’s predicting are based on what it expects the average text in its training set to look like. The loss you’re talking about is how closely its answers match the training set, not how clever the answers sound (though with RLHF it’s different). A model producing better text than what’s in its training set would be penalised for not matching it closely and quickly learn to not do that
Post reply on HN