Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s policies hinder reproducible research on language models

aisnakeoil.substack.com

Post reply on HN
Print view View original

291–300 of 394 posts

Re: OpenAI’s policies hinder reproducible research on language models

#291

Post by segfaultbuserr »

gwd wrote: post #197
danpalmer wrote: post #186

Earlier quoted context omitted.

And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

> I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. [...] If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months [...] Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions

I'm familiar with the "AI safety" movement. For years, many people from that camp are extremely critical of OpenAI and they genuinely believe OpenAI is unleashing something truly dangerous to humanity. One person I knew said that while free and open source is usually important, but due to the unique dangers of AI, it's better to keep AI tech stay in the hands of a small number of monopolists, similar to nuclear non-proliferation. Meanwhile, OpenAI was trying to promote openness - a terrible idea.

Thus, it's indeed a perfect explanation of OpenAI's decision to stop keeping its research in the open. Unfortunately, the problem here is that the "for profit" and "AI safety" explanations are not contradictory, they can simultaneously be true. Just like how Google began as a promoter of the open Web but gradually started to use its market position for its own gain. The same situation exists for OpenAI. "AI Safety" may be the initial motivation, but possibly not for long. After a while, "safety" may be nothing more than an excuse for profit.

Re: OpenAI’s policies hinder reproducible research on language models

#292

Post by anonymousDan »

robertlagrant wrote: post #258
anonymousDan wrote: post #214

Earlier quoted context omitted.

What a bunch of BS. The only reason they are keeping it private is for commercial gain .

I don't really understand this - it's like trying to explain a colleague's behaviour by saying they're doing something so they get their salary. Of course they need to have commercial gain in mind. But you need to be more specific.

From my reading of the parent's comment, they are saying the reason the models are not being made available is because of a fear they will effectively turn into SkyNet - am I being uncharitable?

Re: OpenAI’s policies hinder reproducible research on language models

#293

Post by pffft8888 »

ChatGTP wrote: post #250
ChatGTP wrote: post #164

Earlier quoted context omitted.

It's fine to be hacking, if you're not making billions off the service which people expect some type of stability or baseline performance from, at least that's how I interpret what the parent is saying. Maybe it's easy enough for them to just copy the model, tweak, hack and play with it from there with little interruption. No one really knows at the moment.

Sorry, my writing was crap... I meant to say that now the model is in production, it definitely needs to maintain and or improve performance...

Yes but how will they do that if they don't have a clear understanding. When we build software, we have (or should have) a clear understanding of the various components and, in some cases, like with distributed and mission-critical/military systems, a formal verification/simulation of the system when needed. When we're dealing with emergent behavior, as we have with these large transformers, but no exact understanding of how the behavior is produced and only a limited way of refining/controlling it, I don't think we're in a position to guaratee that refinements in one area won't lead to regressions in other areas or a change in the global characteristics of the system. I mean... we're dealing with complex emergent behavior, at a different scale of complexity than what we have had to deal with so far (in traditional software development) and no mature verification/analysis tools.

Re: OpenAI’s policies hinder reproducible research on language models

#294

Post by godshatter »

scotty79 wrote: post #263
gwd wrote: post #254

Earlier quoted context omitted.

From their technical report [1]: > 2.12 Acceleration > OpenAI has been concerned with how development and deployment of state-of-the-art systems like GPT-4 could affect the broader AI research and development ecosystem.23 One concern of particular importance to OpenAI is the risk of racing dynamics leading to a decline in safety standards, the diffusion of bad norms, and accelerated AI timelines, each of which height…

I really don't understand all those concerns. It's as if people saw a parrot talk for the first time and immediately concluded that they will take over the human civilisation and usher nuclear annihilation upon us because there might be so many parrots and they migh have a hive mind and ... and ... all the wild scenario stemming from the fact you know nothing about parrots yet and have a very little skepticism about…

I don't understand either. An actual AI that could reason about computer code, that understood code well and could create new algorithms and that was smart enough to ask salient questions about what intelligence actually is and that was allowed to hack on it's own code and data store would be something to really worry about.

The worst thing I can worry about with ChatGPT is that someone will ask it for code for something important and not verify it and cause a massively-used system to go down. If it hacked on it's own code and data it would probably in effect commit suicide. It's a "stochastic parrot", as I have heard it called on HN. All my fears have to do with trusting it's output too much.

Re: OpenAI’s policies hinder reproducible research on language models

#295

Post by horns4lyfe »

gwd wrote: post #197
danpalmer wrote: post #186

Earlier quoted context omitted.

And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

So there are serious people out there devoting their time to stopping some imaginary skynet? Is their entire life built around sci fi tropes? Have they ever stepped outside?

Re: OpenAI’s policies hinder reproducible research on language models

#296

Post by [deleted] »

gwd wrote: post #178

If you came here after only reading the headline, you missed what the complaint is actually about: It's not that GPT-4 is closed source. It's that access to `codex` model was pulled with only three days notice, and the model itself was not open-sourced. Since apparently a large number of researchers were writing papers which used that particular model, that means all of those research papers are now non-reproducible.…

[deleted]

Re: OpenAI’s policies hinder reproducible research on language models

#299

Post by z3c0 »

itsaquicknote wrote: post #275
z3c0 wrote: post #242

Earlier quoted context omitted.

I thought I must be going crazy until I saw your comment. This sounds like a bad research practice that probably shouldn't be reproduced to begin with.

Research into systemically important infrastructure cannot be damned because that infrastructure isn't public. It's a cheap moralizing argument to say "pfff, this was predictable". Maybe so, but there isn't an alternative. Much like research on Twitter. Once these companies start to drift into providing what become broadscale social utilities and public services it doesn't matter that they're private. There are(/shou…

I'm going to be frank here, because I know my argument isn't "cheap". When one utilizes OSINT techniques (which using an ML service hosted by a third-party certainly qualifies as), there are baked-in assumptions that

1) this source could go away at any time, and

2) the source is only a reflection of the interests of the third-party, not something to be taken at face value.

No 2 can certainly be the subject of research, but to do so without accounting for No 1 would indicate bad research practices from the jump. For example, they could have (and should have) been snapshotting the outputs, tagged with versions & dates. By the sound of it, the outputs weren't even the subject of research, but were instead propping up the research. That flies in the face of No 2 as well. Let them start over, with better methodology this time.

Re: OpenAI’s policies hinder reproducible research on language models

#300

Post by ben_w »

emptysongglass wrote: post #232
gwd wrote: post #197

Earlier quoted context omitted.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

No one discusses the elephant in the room: who elected these elites to decide what was and wasn't ethical and responsible? Nobody. So who ends up making the ethical decisions? A group of highly privileged SV types insulated from the very real problems, concerns, and perspectives of the ordinary person. This is just more of what humans have been doing over millennia: taking power then telling everyone else it was too…

> who elected these elites to decide what was and wasn't ethical and responsible? Nobody

First: basically every American literally voted for that by repeatedly saying no to the alternative (the communist party) in every American election.

Second: what exactly and specifically are you suggesting here? Because even outside of capitalism, the alternative to "people deciding they personally don't feel it's safe to release a product they created and worked on and know more about than literally anyone else" sounds like actual literal insanity to me.

Post reply on HN