Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s policies hinder reproducible research on language models

aisnakeoil.substack.com

Post reply on HN
Print view View original

341–350 of 394 posts

Re: OpenAI’s policies hinder reproducible research on language models

#341

Post by [deleted] »

sverona wrote: post #327
hamburga wrote: post #306

Earlier quoted context omitted.

^ Thanks for that link. The doomerism is brilliant and clear and imaginative and absolutely worth reading and grappling with. I personally have no good response to how we deal with sufficiently advanced AI’s capacity to trick and manipulate us into doing catastrophically bad things.

His argument is essentially "a superintelligence who is better than us at everything and thinks a quadrillion times faster can do whatever it wants and we are powerless to stop it." Yeah, I could've told you that. If we really are going to create such an intelligence in the next five years, then we had a good run, so long and thanks for all the fish. But that assumption coupled with the security mindset he brings to…

[deleted]

Re: OpenAI’s policies hinder reproducible research on language models

#342

Post by nicky0 »

modriano wrote: post #309
ben_w wrote: post #300

Earlier quoted context omitted.

> who elected these elites to decide what was and wasn't ethical and responsible? Nobody First: basically every American literally voted for that by repeatedly saying no to the alternative (the communist party) in every American election. Second: what exactly and specifically are you suggesting here? Because even outside of capitalism, the alternative to "people deciding they personally don't feel it's safe to releas…

Two notes: 1) less than half of Americans vote in each election (less than 63% if you restrict to the voting-age population, less than 70% if you apply the scummy rules that restrict to the voting-eligible population) And 2) it's a false dichotomy to say that US elections have ever been "whatever we have now VS communism". Maybe you could say socialism was on the ballot all those times Eugene Debs ran for the preside…

The half that don't bother to vote forfeit their right to be counted.

Re: OpenAI’s policies hinder reproducible research on language models

#343

Post by HDThoreaun »

pantalaimon wrote: post #337
HDThoreaun wrote: post #326

Earlier quoted context omitted.

The anti-gmo/nuclear people have no explanation for how things can go wrong. The AI alignment people do. You might not agree with it, but tons of AI researchers, including many at openAI, do.

A nuclear meltdown is much more tangible than a rogue AI somehow taking over the world.

Nuclear meltdowns don’t have the ability to end humanity

Re: OpenAI’s policies hinder reproducible research on language models

#344

Post by bnralt »

pantalaimon wrote: post #337
HDThoreaun wrote: post #326

Earlier quoted context omitted.

The anti-gmo/nuclear people have no explanation for how things can go wrong. The AI alignment people do. You might not agree with it, but tons of AI researchers, including many at openAI, do.

A nuclear meltdown is much more tangible than a rogue AI somehow taking over the world.

Indeed. No matter the likelihood of these things happen accidentally, we at least have the ability to create a situation with nuclear power or GMOs that would kill large amounts of people in the present if that was our goal. We couldn’t create a killer AGI right now even if we wanted to and put a huge amount of resources into it. Even if we made one, we don’t know it would be any more powerful than a human who’s paralyzed from the neck down.

If you use the same assumptions AI alignment folk use for any other tech (“maybe we’ll be able to create a super powerful version of this even though we currently have no clue how to”/“”maybe that hypothetical super powerful version will be able to destroy the world”), they all become extremely dangerous. The alignment crowd usually handles this by only looking at the known issues for most tech today, but then looking at theoretical unknown issues of futuristic tech years from now when it comes to AI.

Re: OpenAI’s policies hinder reproducible research on language models

#345

Post by shagie »

lyu07282 wrote: post #208
dylan604 wrote: post #138

Earlier quoted context omitted.

is there something to be said that seti@home was CPU only? would the GPU give a performance benefit that seti did not have? are people still using the GPUs to mine coins, or is that GPU compute at home available now?

The problem is not compute power the problem is weight and data synchronization. Each iteration or epoch builds on the previous, you either need to run the full model on each node with part of the data and you synchronize every epoch or you run part of the model but then you need to synchronize weights after each iteration. In proof of work mining you don't need to synchronize between each iteration, that's why in mi…

While not likely used here, a fun chip for doing ML - https://www.cerebras.net/product-chip/

From Tom's Hardware: https://www.tomshardware.com/news/cerebras-wafer-scale-engin...

> Power Consumption (System/Chip) 20kW / 15kW

Putting 15kW into one chip is really impressive. The power and cooling for that gets rather interesting.

The piece that reminded me of this is the comment on the bandwidth:

> Fabric Bandwidth 220 Pb/s

Re: OpenAI’s policies hinder reproducible research on language models

#346

Post by UncleEntity »

red1reaper wrote: post #270
UncleEntity wrote: post #225

Earlier quoted context omitted.

Exactly, expecting a company to maintain a project forever (or give it away) just because they were used in research projects is ludicrous. Maybe they did the research when “open” in their name meant something but it has been obvious for quite a while that ship has sailed.

Expecting a company to maintain a project forever or give it away just because they were used in research projects is actually ver reasonable.

Very reasonable if you’re the one doing the research but not if you’re the one subsidizing the maintenance of an obsolete project.

Open sourcing it is its own can of worms because they may be using third-party code they can’t release or, you know, &etc.

Re: OpenAI’s policies hinder reproducible research on language models

#347

Post by skissane »

UncleOxidant wrote: post #338
skissane wrote: post #127

Earlier quoted context omitted.

> However, I'd change your statement a bit: We might look at 2023 as a brief golden age when regular people could access trained parameters (the LLaMA params) and run these models on their own machines (such as with alpaca.cpp). I doubt we'll get access to LLM params again unless some kind of non-profit, actual open source organization is formed to produce them and put them out into the public domain. There are a lot…

Sure, but that's still going to be a commercial product you'll have to pay for. Right now you can run LLaMA (and it's rapidly multiplying fine-tuned descendants) for free. The risk for these startups you describe as working on this as you type is the same thing happening to them that happened to Meta when they released their LLaMA params: they started getting copied all over the place. And it's not clear that Meta ca…

> And it's not clear that Meta can do anything about this. It seems that params aren't copyrightable.

There is a legal argument that they aren’t in the US, but I don’t think that argument has been tested in court yet. Even if the courts uphold that argument-it is likely to fail in other countries, many of which have lower standards for copyrightability than the US does; and it is always possible Congress will respond by creating a new form of IP protection for them. That’s happened before - courts ruled that semiconductor masks weren’t copyrightable, so Congress invented a new “semiconductor mask right” to give copyright-equivalent protection to them. Given the amount of media focus on AI, if courts rule params can’t be copyrighted, very likely Congress invents “AI parameter rights”

Re: OpenAI’s policies hinder reproducible research on language models

#348

Post by glitchc »

gwd wrote: post #197
danpalmer wrote: post #186

Earlier quoted context omitted.

And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

There's way too much hubris in this people. ChatGPT is great, a wonderful tool, and a force multiplier, but it cannot think for itself nor does it want to. We're still a ways away from sentience.

Re: OpenAI’s policies hinder reproducible research on language models

#350

Post by Alifatisk »

gwd wrote: post #254
Alifatisk wrote: post #243

Earlier quoted context omitted.

> I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly re…

From their technical report [1]: > 2.12 Acceleration > OpenAI has been concerned with how development and deployment of state-of-the-art systems like GPT-4 could affect the broader AI research and development ecosystem.23 One concern of particular importance to OpenAI is the risk of racing dynamics leading to a decline in safety standards, the diffusion of bad norms, and accelerated AI timelines, each of which height…

Thank you for the insight! I had no idea so this is an eye opener for me.
Post reply on HN