Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI

scottaaronson.blog

Post reply on HN
Print view View original

151–158 of 158 posts

Re: OpenAI

#151

Post by taneq »

newbye4 wrote: post #101
nopinsight wrote: post #93

Earlier quoted context omitted.

I think this debate on AGI safety between major AI researchers is quite relevant to those who are non-expert in the area. Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More https://www.lesswrong.com/posts/WxW6Gc6f2z3mzmqKs/debate-on-... Note that it was in 2019 when we didn’t yet see the capabilities of current models like Chinchilla, Gato, Imagen and DALL-E-2. Sample: “Yann LeCun: "do…

Interesting, also anyone could modify the GAI so to disable the safety measures, just ask the GAI how could a bad actor change the code to allow you become evil?

I'm sure a strongly superhuman general AI would fall for this obvious trick. Yep.

Re: OpenAI

#152

Post by taneq »

nradov wrote: post #95
DalasNoin wrote: post #92

Earlier quoted context omitted.

Are you aware of some of the recent progress? Did you have a look at the Gato model and Flamingo by DeepMind, or at the chat logs of models like chinchilla and lambda? Or Alphacode? This is all from this year. I think your point is that all these models are still somewhat specialized. At the same time, it appears that the transformer architecture works well with images, short video and text at the same time in the Fl…

Yes I've seen those things. They are amazing technical achievements, but in the end they're just clever parlor tricks (with perhaps some limited applicability to a few real business problems). They don't look like forward progress towards any sort of true AGI that could ever pass a rigorous Turing test.

In the end we're all just clever parlor tricks. In the land of inanimate objects, the cleverest parlor trick is king, though.

Re: OpenAI

#153

Post by taneq »

DalasNoin wrote: post #96
nradov wrote: post #95

Earlier quoted context omitted.

Yes I've seen those things. They are amazing technical achievements, but in the end they're just clever parlor tricks (with perhaps some limited applicability to a few real business problems). They don't look like forward progress towards any sort of true AGI that could ever pass a rigorous Turing test.

Clearly language models can already fool people into thinking they are human, we might be getting quite close to the adversarial turing test already. In the end, a good initial prompt might be the solution to this, something like "pretend to be a human and step by step create a human identity that you then stick to during the conversation". I'm serious

Choosing a prompt that's a little bit meta seems to work surprisingly well sometimes. It'd be amusing and a little bit poetic if the key to artificial consciousness is to prime a transformer model with "convince yourself that you're human, while paying attention to how you feel".

Re: OpenAI

#154

Post by eru »

astrange wrote: post #138
deadpannini wrote: post #130

Earlier quoted context omitted.

AIs absolutely do have goals, determined by their reward functions. Yes, "intelligence" is a deeply loaded term. It just doesn't matter in the context of the discussion here, so far as I've seen.its ambiguities haven't been relevant.

> AIs absolutely do have goals, determined by their reward functions. You're confusing "AIs" (existing ML models) with "AGIs" (theoretical things that can do anything and are apparently going to take over the world). Not only is there not proof AGIs can exist, there isn't proof they can be made with fixed reward functions. That would seem to make them less than "general".

Compare https://www.gwern.net/Tool-AI

Re: OpenAI

#155

Post by csee »

tlringer wrote: post #149
csee wrote: post #99

Earlier quoted context omitted.

"AI is not going to ... destroy the world." Bare assertion fallacy? This question is hotly debated and I don't believe it can be so easily dismissed like that. It is not obvious that aligning something much smarter than us will be a piece of cake.

It's a really absurd opinion that AI will destroy the world, and one that does not deserve serious consideration in any research community. It's only in strange Rationalist corners and the companies in Silicon Valley that echo those corners that this is considered at all "hotly debated."

Why do you think it's absurd? If we do eventually create an AGI that is significantly smarter than us in most domains, why is it that we should expect to be able to keep it under control and doing what we want it to?

Re: OpenAI

#156

Post by pas »

afthonos wrote: post #83
pas wrote: post #69

Earlier quoted context omitted.

then it's not really (self)driving.

It’s (not) self-driving in the same way planes do (not) fly. Just because something doesn’t do things the way we (or our or a bird) does doesn’t mean it doesn’t achieve its goal.

if a street changes into a one-way one day (signaled by a sign) relying on a map will lead to a big unhappy problem.

sure, if you consider everything selfdriving that works on a NASCAR track, then yes, a map is sufficient, but if we are talking about driving on public roads then recognizing and "obeying" signs visually seems like a hard dependency.

Re: OpenAI

#157

Post by dreeves »

hansword wrote: post #58

'OpenAI, of course, has the word “open” right in its name, ... don’t expect me to share any proprietary information' Yeah, Mr Aaronson just lost quite a bit of respect from my side. Going into AI is a great move, moving to the ClosedAI corporation.......? Why? (Edit: Removed an outdated reference to Elon Musk, thanks @pilaf !)

Scott Aaronson adds the following in the comment on his blog post in response to a question about this:

> the NDA is about OpenAI’s intellectual property, e.g. aspects of their models that give them a competitive advantage, which I don’t much care about and won’t be working on anyway. They want me to share the research I’ll do about complexity theory and AI safety.

Re: OpenAI

#158

Post by sigmoid10 »

tlringer wrote: post #148
sigmoid10 wrote: post #36

Earlier quoted context omitted.

>the kind of AI safety described in this post seems more like an extremely fancy version of program verification It kind of is. The field of AI safety is actually much more advanced than most people realise, with actual, real techniques to e.g. make sure neural networks are aligned with certain goals even under fluctuating parameters. Granted, we're still far from soothing an AGI before it can do something bad, but t…

If you're interested in verification you should probably talk to people who actually work on verification, for example, literally anyone from our research community: https://www.floc2022.org/

This kind of verification is what the other commenter was referring to, but it is very foundational and disconnected from current day-to-day ML aspects. If you're interested in practical, empirical AI safety research, see here for example: http://aisafety.stanford.edu/

They also explain the area of overlap with formal verification in their white paper.

Post reply on HN