Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s policies hinder reproducible research on language models

aisnakeoil.substack.com

Post reply on HN
Print view View original

301–310 of 394 posts

Re: OpenAI’s policies hinder reproducible research on language models

#301

Post by allemagne »

selfhoster11 wrote: post #241
zarzavat wrote: post #181

Earlier quoted context omitted.

ChatGPT-4 is definitely slower than GPT-3.5 (and way slower than 3.5-turbo). What could be the reason for that other than much larger parameter count? I agree that the capabilities seem overhyped. In my subjective experience, 4 seems a little better than 3.5 but not by a huge amount. We just have OpenAI’s cherry-picked word that it‘s this incredible advance.

I disagree. It does much, much better on selected tasks. I cannot quite figure out how to describe what the difference "feels" like, but the performance is sometimes markedly different when feeding ChatGPT-3.5 and ChatGPT-4 the same prompt.

One task that ChatGPT-3.5 is hilariously bad at is reversing strings (both words and pseudorandom input). It seems to have only a vague concept of what that means, even if I try to hold its hand through the process. Maybe some prompt engineering can get it to succeed on anything longer than four letters.

ChatGPT-4 meanwhile seems to have no issue with this at all.

Re: OpenAI’s policies hinder reproducible research on language models

#302

Post by dmix »

lalos wrote: post #256
gwd wrote: post #197

Earlier quoted context omitted.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

> Here's one example: GPT-4 was actually done back in August of last year. If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. They did that, that's how Reid Hoffman got early access to write his book, that's how Microsoft got access to start working on Bing/ChatGPT4 for a cool $10 billion and that's how countless of other got early access. Got the m…

Besides wasn't releasing GPT3 supposed to have caused major harm to society? Which is why they held off for so long. Still waiting for evidence of that harm (mass fake news, Google being ruined by even more low ranking spam sites, etc).

It must be nice thinking that a small group withholding the keys R&D (for a short while until other R&D groups catch up) will somehow help the problem. Do these few months to a year really provide much value in finding ways to stop the "AI apocalypse"? What real work are they doing to prevent it in those few months? More philosphizing and high level analysis?

It might work for messaging/marketing that they are being "careful" but I'm not convinced this is tangible. Seems as arrogant and naive as most AI ethics stuff I read.

Re: OpenAI’s policies hinder reproducible research on language models

#303

Post by dmix »

jrochkind1 wrote: post #286
gwd wrote: post #197

Earlier quoted context omitted.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

Okay, but their actions are _not_ stopping AI research, they are doing plenty of AI research internally. They're just hindering competitors and non-profit researchers. I suppose you could make an argument that nobody can be trusted to do AI research as responsibly as them, so that's why they should not share anything and should hinder others' research... but it kind of looks like plain old nothing-to-see-here profit-…

"Virtue signaling" is overused but highly relevant here. Absent some proof they've done anything at all to prevent an AI takeover (which surely would have to be open source to be valuable too right?).

Re: OpenAI’s policies hinder reproducible research on language models

#304

Post by IMTDb »

gwd wrote: post #197
danpalmer wrote: post #186

Earlier quoted context omitted.

And those suggestions would be very in-line with the original purpose of OpenAI. A purpose they are now actively hindering in the name of profit.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

> Most of those individuals have absolutely nothing commercial to gain from stopping AI research.

Most individual trying to stop vaccine research and rollout have nothing to gain from it; that does not mean they are right. Do not conflate action and intention.

Re: OpenAI’s policies hinder reproducible research on language models

#305

Post by gjm11 »

Torifyme12 wrote: post #227
sverona wrote: post #223

Earlier quoted context omitted.

It's possible, but the views of the AI alignment community so far as I can tell are being skewed way too far towards nihilistic doomerism by the influence of Yudkowsky, who apparently believes that we're all gonna die in a few years and there's nothing anyone can do to stop it. [0] [0] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

Yudkowsky Man how the fuck does that guy keep popping up in the most random places starting fights?

Thinking and pontificating about AI safety is literally his job, and Less Wrong is a thing he founded, so whatever else Yudkowsky pontificating about AI safety on Less Wrong might be, it isn't "popping up in the most random places".

Re: OpenAI’s policies hinder reproducible research on language models

#306

Post by hamburga »

sverona wrote: post #223
oefnak wrote: post #211

Earlier quoted context omitted.

You sound rational. Do you not agree with the possibility of AI doom soon?

It's possible, but the views of the AI alignment community so far as I can tell are being skewed way too far towards nihilistic doomerism by the influence of Yudkowsky, who apparently believes that we're all gonna die in a few years and there's nothing anyone can do to stop it. [0] [0] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

^ Thanks for that link. The doomerism is brilliant and clear and imaginative and absolutely worth reading and grappling with. I personally have no good response to how we deal with sufficiently advanced AI’s capacity to trick and manipulate us into doing catastrophically bad things.

Re: OpenAI’s policies hinder reproducible research on language models

#307

Post by malwrar »

jillesvangurp wrote: post #196
atleastoptimal wrote: post #32

I understand any individual's company anti-competitive measures. OpenAI looks at Google the same way Apple looked at IBM in the 80s. What I'm worried about is a lot of the talk about guarding models, public safety and misuse of models will end up leading every big company to pull public access of their APIs. We might look at 2022-2023 as a brief golden age when regular people could use stuff like GPT-4 before it was…

You are assuming openai is going to end up with a monopoly on all this. IMHO the opposite is going to happen. There are going to be a multitude of companies and researchers competing on outdoing what they are doing in terms of quality, cost, and use cases. If big companies put a straight jacket in place to limit access, constrain usage, etc., that just creates the opportunity for others to step up and grab some marke…

I personally expect that regulation or collusion among big tech players (e.g. the suppression of parlor) will prevent the average person or company from having the legal or practical ability to amass the compute power and dataset necessary to train a competing LLM (or future arch).

No one really seems to know if OpenAI’s use of copyrighted materials like published works and open source code for training its LLM is legal. I can easily see a future where use of copyrighted works like this simply can’t be repeated legally, and the compute power necessary to do it as an individual is made inaccessible. This especially if the resulting model is made open to everyone, it’s such a wild cultural shift to me seeing tech nerds advocating against democratization of this tech due to personal doomsday fantasies. The sheer number of people who exist in the community and are obsessed with alignment and ethics will provide plenty of ideas on practical constraints the non-technical powerful could impose to make this real.

Re: OpenAI’s policies hinder reproducible research on language models

#308

Post by ben_w »

bnralt wrote: post #288
gwd wrote: post #197

Earlier quoted context omitted.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

The danger the AI alignment folk are afraid of is completely impossible with current tech, but they want to put up barriers because we have no idea what future tech might look like and there’s the possibility some future advance could be very dangerous. When anti-GMO or anti-nuclear folk used this same standard to put up barriers to research into nuclear or GMO research, they get lambasted for being anti-science, but…

The only reason I have to think it's impossible for current AI to pay someone to help it bootstrap itself into other hardware is because OpenAI researchers tried to get it to do exactly that and reported that it failed.

The only reason I'm confident other AI public models won't determine highly potent novel neurotoxins is that the company who made the AI model which did exactly that thing when they flipped a bit from "least dangerous" to "most dangerous" were absolutely terrified and presumably kept enough away from the public domain.

The only reason I'm even hopeful that DNA-on-demand companies keep a watch out for known pathogens is the SciFi about such things going wrong might make them at least try to not do that.

Unthinkable man made horrors have been with us for an extremely long time; AI isn't new in this regard, but as intelligence is the human superpower, even in the context of AI that have no agency of their own, it can elevate stupid arseholes to the level of dangerous arseholes.

Re: OpenAI’s policies hinder reproducible research on language models

#309

Post by modriano »

ben_w wrote: post #300
emptysongglass wrote: post #232

Earlier quoted context omitted.

No one discusses the elephant in the room: who elected these elites to decide what was and wasn't ethical and responsible? Nobody. So who ends up making the ethical decisions? A group of highly privileged SV types insulated from the very real problems, concerns, and perspectives of the ordinary person. This is just more of what humans have been doing over millennia: taking power then telling everyone else it was too…

> who elected these elites to decide what was and wasn't ethical and responsible? Nobody First: basically every American literally voted for that by repeatedly saying no to the alternative (the communist party) in every American election. Second: what exactly and specifically are you suggesting here? Because even outside of capitalism, the alternative to "people deciding they personally don't feel it's safe to releas…

Two notes:

1) less than half of Americans vote in each election (less than 63% if you restrict to the voting-age population, less than 70% if you apply the scummy rules that restrict to the voting-eligible population) And 2) it's a false dichotomy to say that US elections have ever been "whatever we have now VS communism". Maybe you could say socialism was on the ballot all those times Eugene Debs ran for the presidency, but there hasn't ever been a communist on the ballot that I'm aware of. Also, it sounds like you would struggle to define communism if pressed.

Regarding your second loose point, the US restricts the sale of a lot of products to the public (eg nuclear weapons, biological weapons, raw milk, copyrighted works you don't hold the copyright to, etc). Personally, I think it's pretty reasonable to restrict the sale of some things, even if the potential sellers know a lot about the product.

Re: OpenAI’s policies hinder reproducible research on language models

#310

Post by mrandish »

segfaultbuserr wrote: post #291
gwd wrote: post #197

Earlier quoted context omitted.

I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. To you it looks like they're being overly careful and paranoid, perhaps as an excuse to set up a monopoly silo to extract money. But a lot of the people the OpenAI researchers work closely with -- people deep in the "AI alignment" community -- are telling them that they're being wantonly reck…

> I think what most of the people here are missing is how big, how paranoid, and how influential the "AI alignment" movement is. [...] If their goal was to maximize profit, the obvious thing to do would be to release API access to it as soon as possible. But instead, they purposely delayed release for eight months [...] Go lurk on alignmentforum.org for a while, and you'll have a different perspective on OpenAI's dec…

> free and open source is usually important, but due to the unique dangers of AI...

Sadly, cherished principles often perish on the horns of "But this time it's different."

> the "for profit" and "AI safety" explanations are not contradictory

Indeed, they can reinforce each other into a runaway feedback loop. Once you buy into an all-encompassing mission of preventing apocalypse, maintaining perspective or proportionality become almost impossible. The moral hazard of not pursuing almost all available measures justifies taking $10B of MSFT's money to fund the defense of humanity. Add to this the ego-stoking existential importance of such a "noble cause", the global media attention and the social elevation in the tight-knit, closed-circle of the AI Alignment community and you've got the perfect drug.

Given the intense forces shaping the worldview of the "AI Safety Noble Warriors", it's reasonable for the rest of us to question their objectivity and suspect claims of "we are keeping this from you for your own good."

Post reply on HN