Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

Post reply on HN
Print view View original

261–270 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#261

Post by emptysongglass »

josephg wrote: post #148
SuperLynxDeluxe wrote: post #128

Earlier quoted context omitted.

That really depends on who you asked. Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap. Deep learning is to neural networks what GPT4 is to transformers. It's a lot bigger, enabled by better hardware and more data, but fundamentally not a new technology. There needs to be a lot more leaps for AGI, language models are not the holy grail, but another tool like reinforcement l…

I think the leaps required are: - Giving gpt4 short term memory. Right now it has working memory (context size, activations) and long term memory (training data). But no short term memory. - Give it the ability to have internal thoughts before speaking out loud - Add reinforcement learning. If you want to write code, it helps if you can try things out with a real compiler and get feedback. That’s how humans do it. I…

You're guessing, which is a problem, and gives you license to conclude that:

> I don’t think agi will be that far away.

To,

> Give it the ability to have internal thoughts before speaking out loud

, you have no idea what that means technically because no one knows how internal thinking could ever be mapped to compute. No one does, so it's ok to not know, but then don't use it as a prior to guess.

Some have maybe seen this before, if you haven't I'll say it again: compute ≠ intelligence. That LLMs offer convincing phantasms of reasoning is what gives the outside observer the idea they are intelligent.

The only intelligence we understand is human intelligence. That's important to emphasize because any idea of an autonomous intelligence rests on our flavor of human intelligence which is fuelled by desire and ambition. Ergo, any machine intelligence we imagine of course is Skynet.

Somebody in some other conversation here pointed out the paperclip optimizer as a counterpoint but no, Bostrom makes tons of assumptions on his way to the optimizer, like that an optimizer would optimize humans out of the equation to protect its ability to produce paperclips. There's so many leaps of logic here which all assume very human ideas of intelligence like the optimizer must protect itself from a threat.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#262

Post by joe_the_user »

calamari4065 wrote: post #260
joe_the_user wrote: post #221

Earlier quoted context omitted.

That's not the (only) problem, it's that no one can describe what AGI will really look like outside of vague and circular definitions The basic description is "a computer system that can do all relevant things a human being can do". And sure, we don't have a very codified definition of what those things are but we, human beings, exist and so even if we can't define our own capacities, we have an strong idea they exis…

> The implicit argument is that one can't engineer something one doesn't understand. That's what machine learning is . We build these systems we don't fully understand, and set them on solving problems we don't understand. It's revolutionary because it lets us solve problems we don't understand. We don't really fully understand what they do or how, but it does still work.

That's what machine learning is

Technically that's what deep learning accepts. Other forms of machine learning have greater or lesser degrees of explicit statistical basis.

It's revolutionary because it lets us solve problems we don't understand

It's certainly a change! I would submit that an approach like that also is often limited in it's understanding of to what degree it has solved a given problem. That's not too bad when you have human verifying the end result but can problematic if one turns a machine of this sort loose to make it's own decisions.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#263

Post by golol »

theptip wrote: post #182
golol wrote: post #161

Earlier quoted context omitted.

No, then you start worrying when the technology is there. There is no point in worrying about it now when it is fiction. If AGI/ASI does not arise in some kind of uncontrollable "foom", you can always examine what you are dealing with right now and act accordingly. If AGI is achievable in a 5 years takeoff scenario then we have plenty of time and steps along the way fo study what we have and examine its danger. Right…

If AGI is achievable in 5 years, then we do need to worry now, because we currently have no idea how to solve problems like prompt hijacking or how to durably convey our values to these systems. > There is no point in worrying about it now when it is fiction. I find this sort of epistemic nihilism completely baffling. Just because we have wide error bars, doesn’t mean we have to give up on trying to steer the future.…

It's not nihilism, I just expect absolutely nothing productive to come out of an attempt to regulate and control a technology that doesn't exist yet. Please show me a single result of alignment research that has real world applicability and is not just a variant of "train your model on aligned data so it is aligned". And the latter form of alignment success, RLHF and instruction finetuning, came after the base model capability was unlocked.

If you are doing any sort of challenging intellectual endeavour you need one of two things in order not to be inevitably wrong: you either use rigor as in math or the scientific method, or you do engineering with real things that you can test and play around with.

If neither of these two are giben then your field is likely to produce decades worth of junk results and predictions such as it happens in economics, sociology, psychology, and especially philosophy. You could fund a billion dollar 10 year program with economists in order to prevent the next big market crisis and they would not produce any valuable work.

I see AGI alignment research as being kind of in this position: Rigorous mathematical results are impossible or impractical (theoretical results about perfect actors maximizing utility functions or sth are unlikely to apply on the real world), and if you do not have AI models of that capability level available, you can also not engineer, test, and play around with your ideas. Hence I lots and lots of papers and theories will be produced and eventually the real world will show what was correct, but no valuable predictions will have been obtained.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#264

Post by consumer451 »

mr_toad wrote: post #252
chpatrick wrote: post #206

Earlier quoted context omitted.

What I mean is if context length wasn't a limit, you could have your own AI agent that remembers that article forever, along with everything else you've ever discussed with it. If it generated text on its own, without human interaction, and developed its "personality", then you could say it's more like what we consider an intelligent being.

Context length is limited by the length of articles in the training data. You couldn’t use infinite context lengths unless the LLM had been trained on infinity long articles. Context lengths aren’t arbitrary limits - they reflect the ability (and limits) of the model to actually reason about lengthy topics. If you want longer context lengths you’ll need more training data, larger models, and a lot more money.

Thank you.

So in my very uniformed mind.. "AGI," at the very least, would require real-time retraining... correct?

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#265

Post by theptip »

golol wrote: post #263
theptip wrote: post #182

Earlier quoted context omitted.

If AGI is achievable in 5 years, then we do need to worry now, because we currently have no idea how to solve problems like prompt hijacking or how to durably convey our values to these systems. > There is no point in worrying about it now when it is fiction. I find this sort of epistemic nihilism completely baffling. Just because we have wide error bars, doesn’t mean we have to give up on trying to steer the future.…

It's not nihilism, I just expect absolutely nothing productive to come out of an attempt to regulate and control a technology that doesn't exist yet. Please show me a single result of alignment research that has real world applicability and is not just a variant of "train your model on aligned data so it is aligned". And the latter form of alignment success, RLHF and instruction finetuning, came after the base model…

> an attempt to regulate and control a technology that doesn't exist yet.

I’m not even proposing regulation. Just trying to understand the technology. You say it doesn’t exist yet, but many AI researchers (including Sutskever and Hinton, pioneers that know as much about the state of the art as anyone alive) think AGI could be achieved by scaling up the current architectures.

If there is a non-zero probability of AGI built atop transformers, we must invest more in understanding how transformers actually work. Even if it’s not that likely, it’s worth spending a lot more than we do now as a hedge. And I believe that if transformers do not end up being anything to do with AGI, it is likely that we will still carry over some expertise from catching up and understanding them.

For papers that are actually building our understanding here, see https://transformer-circuits.pub/. For example I think “Towards Monosemanticity”[1] was fairly widely considered to be a big step forward in the field.

There have also been meaningful results around steering models, which again relies upon being able to decode what features are actually encoding specific concepts.

I’m here for the idea that most papers are bunk, but isn’t that science in general? Really hard to pick good projects and researchers to invest in ahead of time, and so you kind of have to spray and pray at some level. Still worth it on net.

[1]: https://transformer-circuits.pub/2023/monosemantic-features/...

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#266

Post by dartos »

calamari4065 wrote: post #260
joe_the_user wrote: post #221

Earlier quoted context omitted.

That's not the (only) problem, it's that no one can describe what AGI will really look like outside of vague and circular definitions The basic description is "a computer system that can do all relevant things a human being can do". And sure, we don't have a very codified definition of what those things are but we, human beings, exist and so even if we can't define our own capacities, we have an strong idea they exis…

> The implicit argument is that one can't engineer something one doesn't understand. That's what machine learning is . We build these systems we don't fully understand, and set them on solving problems we don't understand. It's revolutionary because it lets us solve problems we don't understand. We don't really fully understand what they do or how, but it does still work.

We do fully understand how, and why, they work. That’s how we can build and optimize them.

We can and do explicitly explain their inner workings in math.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#267

Post by goldenkey »

ClumsyPilot wrote: post #173
RandomLensman wrote: post #160

Earlier quoted context omitted.

The US poured a lot of money into the SST, too, it wasn't just the French and the British. A lot of people thought that the future is supersonic as people did want the speed increase when moving to jet planes. Conversely, maybe AGI is not what is wanted in the end anyway.

> Conversely, maybe AGI is not what is wanted in the end anyway. I dont think it is, if it is trully intelligent, it will probably have rights, so you cant treat it as a slave. And it will likely not be controllable. Both of these mean it will not be commercially progotable, any more than having children is profitable.

The animals we farm are intelligent and yet we still override their desires for their bodies with our own, for slaughter.

Hell, even plants are very sophisticated (intelligent) conscious biochemical programs which have an aversion to being wounded: https://www.theguardian.com/environment/2023/mar/30/plants-e...

Consciousness is the universe evaluating if statements. When we pop informational/energetic circuits in order to preserve our own bodies and/or terraform spacetime, we're being energetic enslavers.

We should stick to living off sustainable/self-sustaining sources - fruit, seeds, nuts, beans, legumes, kernels, and cruelty-free dairy/eggs/cheese. The photons that come from the sun are like its fruit. No circuits needing popping.

Notice that all information systems are conscious in their degrees of freedom, even mechanical ones: https://youtu.be/mcedCEhdLk0?si=oXhr7bgg5UkPLLvg

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#268

Post by youarelabor »

mr_toad wrote: post #259
youarelabor wrote: post #197

Earlier quoted context omitted.

We already have procedures for getting the 'right' people, as far as it can be determined, into positions of power/decision-making authority. Unelected greedy fuckers like Sam Altman should be laughed at and thrown out of power immediately . I would rather have sclerotic legislators, preferably drawn from the Millennial and Zoomer generations, than the greedy fuckers like Paul Graham and Sam Altman pretending they ar…

Who would invest in a business where the executives were elected by non-owners and had conflicting interests? What you’re arguing for is the dictatorship of the proletariat.

[flagged]

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#269

Post by archargelod »

dragontamer wrote: post #231
jdmoreira wrote: post #190

Earlier quoted context omitted.

Sounds like something really simple to define and grasp. What's wrong with, for example: "A software that beats most humans on most cognitive tasks". People make it harder than it needs to he IMO

We've had computers that beat humans in differential equations, automated theorems, arithmetic, aiming a gun and more in the 1970s. Even search was considered an AI before Google, but today is just a thing computers are and always have been faster than humans at. Even before ChatGPT, lawyers relied upon computers to double-check citations. Even in programming, I expect that compilers and IDEs check for the most commo…

That's why it's Artificial *General* Intelligence.

We knew 30 years ago that computer can beat human in the field it was engineered for.

But AGI is good at anything you throw at it, including building a better AI. Now give it agency and that's a recipe for a disaster.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#270

Post by pixl97 »

emptysongglass wrote: post #261
josephg wrote: post #148

Earlier quoted context omitted.

I think the leaps required are: - Giving gpt4 short term memory. Right now it has working memory (context size, activations) and long term memory (training data). But no short term memory. - Give it the ability to have internal thoughts before speaking out loud - Add reinforcement learning. If you want to write code, it helps if you can try things out with a real compiler and get feedback. That’s how humans do it. I…

You're guessing, which is a problem, and gives you license to conclude that: > I don’t think agi will be that far away. To, > Give it the ability to have internal thoughts before speaking out loud , you have no idea what that means technically because no one knows how internal thinking could ever be mapped to compute. No one does, so it's ok to not know, but then don't use it as a prior to guess. Some have maybe seen…

At the end of the day intelligence guesses. If it doesn't guess it's just an algorithm.

Now, if you step beyond your biases that we cannot make intelligent machines and instead imagine you have two intelligent machines that are pitted against each other. These could be anything from stock trading applications to machines of war with physical manifestations. If you want either of these things to work they need to be protected against threats, both digital or kinetic. To think AGI systems will be left to flounder around like babies is very strange thinking indeed.

Post reply on HN