Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s policies hinder reproducible research on language models

aisnakeoil.substack.com

Post reply on HN
Print view View original

381–390 of 394 posts

Re: OpenAI’s policies hinder reproducible research on language models

#381

Post by ben_w »

HDThoreaun wrote: post #343
pantalaimon wrote: post #337

Earlier quoted context omitted.

A nuclear meltdown is much more tangible than a rogue AI somehow taking over the world.

Nuclear meltdowns don’t have the ability to end humanity

No, but they can really mess up property prices in the area. Lotta people care about that.

Also people are really bad with the relative scale of different big things, so "mess up city" and "mess up planet" come across similarly in people's heads. (This is also why people might try to argue against immigration by saying "America is full" — their city is a bigger part of their world view than is, say, rural Montana).

(I am not a huge fan of nuclear power, but I'm also not any kind of opponent; for AI, I can see many ways it might go wrong, but I don't know how to guess at the probability-vs-damage distributions of any of them).

Re: OpenAI’s policies hinder reproducible research on language models

#382

Post by dizhn »

nunodonato wrote: post #373
inquist wrote: post #253

Earlier quoted context omitted.

I searched and didn’t find any identical prior comment

https://www.reddit.com/r/OpenAI/comments/11xbe9o/comment/jd2...

Is it a bot? were you able to figure anything out?

Re: OpenAI’s policies hinder reproducible research on language models

#383

Post by nunodonato »

dizhn wrote: post #382
nunodonato wrote: post #373

Earlier quoted context omitted.

https://www.reddit.com/r/OpenAI/comments/11xbe9o/comment/jd2...

Is it a bot? were you able to figure anything out?

no idea, but its funny how it/he/she fetched the comment a similar reddit discussion and pasted it here...

Re: OpenAI’s policies hinder reproducible research on language models

#384

Post by ugh123 »

lionkor wrote: post #363
Mike_12345 wrote: post #162

Earlier quoted context omitted.

Answer from GPT-4: "This question seems to be intentionally nonsensical or is using unfamiliar terminology. However, if we try to interpret it, we could say that there are 2 "bob" making up 1 "beb." In this case, there would be 0.5 "beb" per "bob." Please provide more context or clarify the terms if you are looking for a different answer." Answer from GPT-3.5 (subscription version, not free): "If 2 bob are a beb, the…

Cool, but sadly, as I said, it did not give a very useful answer. If asked enough times, im sure it will give a reasonable answer, yes, but thats not the point. GPT4s answer is interesting, though

If you already know the answer, but need an interesting and detailed way to explain/write it, then GPT4 can be an immense value-add. Even if you have to do some fact checking afterwards.

Re: OpenAI’s policies hinder reproducible research on language models

#385

Post by MichaelZuo »

nunodonato wrote: post #383
dizhn wrote: post #382

Earlier quoted context omitted.

Is it a bot? were you able to figure anything out?

no idea, but its funny how it/he/she fetched the comment a similar reddit discussion and pasted it here...

So this 'ar9av' is a bot reposting comments or a troll?

Wonder if dang could take a look at this.

Re: OpenAI’s policies hinder reproducible research on language models

#386

Post by jiggywiggy »

KyeRussell wrote: post #191
jiggywiggy wrote: post #145

Earlier quoted context omitted.

Its not interesting. It's a hack to have a don't be evil vibe and keeping the name "open" while they go against their own foundational principles.

You aren’t providing any sort of valuable insight here. This is more indicative of your priors than anything else. Everyone has heard this argument. The people that believe it, believe it. The people that don’t, don’t.

The initial goal was to make ai available to everyone. In the process of getting enough funds to build their vision they gave it to Microsoft.

Re: OpenAI’s policies hinder reproducible research on language models

#387

Post by modriano »

ben_w wrote: post #380
modriano wrote: post #375

Earlier quoted context omitted.

You misunderstand. Saying > "the rejection of communism is why rich people get to own and control businesses". is as wrong as saying "the rejection of [pastafarianism | soccer/futbol | anarchocapitalism | mandatory left-handedness | manual transmission cars | etc] is why rich people get to own and control businesses". Communism and the communist party have never been part of the question. The closest "Communism" came…

You're arguing against an imaginary totem instead of what I actually wrote. > Communism and the communist party have never been part of the question De facto/de jure. Nobody wants it (de facto), I've demonstrated by linking to the actual parties that de jure it's totally been an option. Those parties I linked, you could have voted for, but y'all didn't. Given I've explicitly said I'm not talking about dem/rep culture…

> Those parties I linked, you could have voted for, but y'all didn't.

I've voted in every national election (primary and general) in the past 15 years, and those parties have never been on the ballot. It takes a massive amount of money to run successful campaigns, and political donors make a big difference in choosing which candidates (and therefore which platforms) people get to vote on. This significantly biases the ideological space and makes it invalid to look at the outcomes of elections and draw conclusions about vague ideas that were nowhere near making it onto any major party candidate's platform.

I'm not a proponent of communism; I'm a big fan of property rights, but in any case, my core allegiance is to the scientific method and to seeing reality clearly. You really want to reach a specific conclusion but the claims you're using to build your path to that conclusion are not factual. You should start from a close inspection of the actual processes and behaviors of systems, rather than starting with your conclusion and trying to cobble together a case for your conclusion.

Re: OpenAI’s policies hinder reproducible research on language models

#388

Post by itsaquicknote »

dahart wrote: post #334
itsaquicknote wrote: post #311

Earlier quoted context omitted.

Replying to both responses because they're all good points. My argument boils down to the fact that some private companies end up becoming social utilities and once that happens, the rules (should) change as part of the social contract which means, yeah, they can't simply "pull the rug". The research is important precisely because its into systemically significant systems. I get that it's difficult to define the line…

No matter how you define it, or whether people even agree companies should be obligated to provide certain public services, we are just nowhere near that line yet in this case, net even remotely close. It’s hand-wavy to say it’s important, but this is all brand new, there are only a handful of researchers involved, the critical mass to justify what you’re suggesting does not yet exist, it won’t for some time, and the…

Agree to disagree.

Re: OpenAI’s policies hinder reproducible research on language models

#389

Post by usrbinbash »

gwd wrote: post #354
usrbinbash wrote: post #289

Earlier quoted context omitted.

> Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's decisions. No I won't, because the arguably most successful way of detecting, preventing and/or fixing problems with almost all complex systems, is to have as many eyeballs on them as possible. This has been known in software engineering for quite some time: "Given enough eyeballs, all bugs are shallow." -Eric S. Raymond,…

And you're so sure that this maxim applies to AI alignment, that you're not interested in even hearing what people actually working in the AI field might have to say? (To post on alignmentforum.org, you actually have to demonstrate that you are actively working in AI research.) AND , you're so certain that it applies, that you're willing to potentially risk the fate of the entire human race on it? I wasn't actually s…

> Before you make policy recommendations based on ideology

Ideologies are belief systems. The fact that open sourcing something is a good way to find errors in complex systems is a proven fact.

AI isn't magical in that regard. It is a complex system, and experience with such systems, from economics, to climate mechanisms, to software, teaches us that predictions about them, including error detection, risk management and fixing problems, works the better the more people have a chance to look at its internal workings.

Re: OpenAI’s policies hinder reproducible research on language models

#390

Post by gwd »

usrbinbash wrote: post #389
gwd wrote: post #354

Earlier quoted context omitted.

And you're so sure that this maxim applies to AI alignment, that you're not interested in even hearing what people actually working in the AI field might have to say? (To post on alignmentforum.org, you actually have to demonstrate that you are actively working in AI research.) AND , you're so certain that it applies, that you're willing to potentially risk the fate of the entire human race on it? I wasn't actually s…

> Before you make policy recommendations based on ideology Ideologies are belief systems. The fact that open sourcing something is a good way to find errors in complex systems is a proven fact. AI isn't magical in that regard. It is a complex system, and experience with such systems, from economics, to climate mechanisms, to software, teaches us that predictions about them, including error detection, risk management…

> The fact that open sourcing something is a good way to find errors in complex systems is a proven fact.

Is your evidence for this anything other than anecdotal? That ESR quote was given after one particular LKML interaction back in the 90's. And sure, sometimes an interaction like that happens. But just as often I've seen an email or bug report to an open-source project get completely lost. Not to mention that 1) new security-related bugs are still introduced into Linux, despite the number of eyeballs looking at them 2) people are still finding security-related bugs in Linux which are over a decade old. (Not to pick on Linux here -- but Linux probably has the most eyeballs, so according to this theory it should have the least bugs.)

Asserting that open-sourcing has always reduced bugs in all software isn't supported by those anecdotes; you'd need to do some sort of actual study comparing various types of development.

> AI isn't magical in that regard. It is a complex system, and experience with such systems, from economics, to climate mechanisms, to software, teaches us that predictions about them, including error detection, risk management and fixing problems, works the better the more people have a chance to look at its internal workings.

What "AI alignment" people are saying is that from a risk management perspective, AI is different. Namely, the fear is that we'll get an AI which is much more intelligent than us -- an AI intelligent enough to 1) gain a technological superiority over us 2) anticipate and counter anything we could try to do to stop it; and that this AI might end up with its own random "paperclip-maximizing" plans, and care no more about us than we care about ants; and that it might come into being before we have any idea how dangerous it is.

The worst think that can happen with a bug that gets into the Linux kernel is you may lose some data, or some other human steals some of your secrets. The worst thing that can happen from an AI alignment catastrophe is the extermination of the human race.

Now, maybe those fears are overblown. Or maybe you're right, that open-sourcing everything would be the best way to avert an AI catastrophe in any case. But to assert it's true, based on some thing a random guy said after a random email discussion 25 years ago, without even bothering to engage with what people actively working in the AI field are saying, certainly is an ideology.

Post reply on HN