271–280 of 475 posts
Earlier quoted context omitted.
The whole point is that no one knows how to engineer these principles into a model, and no one has a good plan for doing so either.
Not to mention that having "principles" is going to handicap you in a competitive environment where not being on top means you might as well be last.
Yudkowsky's fear-mongering about sci-fi Skynet is also unmoving to me (for metaphysical reasons I won't go into here) however, his position is at least logically consistent, as Aaronson points out.
These machines can already talk and think better than many humans, and the ratio (of mentally sub-computer to super-computer humans) will only go up. Ergo, the only intellectual problem left is how to use them.
I've been banging on about this for a couple of weeks now, so apologies to those who've seen it before, please read "Augmenting Human Intellect: A Conceptual Framework" SRI Summary Report AFOSR-3223 by Douglas C. Engelbart, October 1962 https://dougengelbart.org/pubs/augment-3906.html
And then at least skim "An Introduction to Cybernetics" by W. Ross Ashby. http://pespmc1.vub.ac.be/books/IntroCyb.pdf The thing you're looking for is "intelligence amplifier". Engelbart references it, the book explains all the math you need to understand and build one.
The next piece of the puzzle is something called "Neurolinguistic Programming" (you may have heard of it, some people will tell you it's pseudoscience, ignore those people, they mean well but they just don't know what they're talking about.) It turns out there's a rigorous repeatable psychology that more-or-less works. It's been under development for about half a century.
The particular thing to look at is a specific algorithm called "Core Transformation Process" ( https://www.coretransformation.org/ ) which is kind of like the "Five Why's" technique but for the mind. It functions to reduce yak-shaving and harmonize intent.
Anyway, the upshot of it all is that these machines can be made into automatic perfect therapists, using off-the-shelf technology.
Earlier quoted context omitted.
Not at all off topic because any slow down on public research just lets all such shadow actors get even further ahead. We have to guess that top level intel agencies probably had gpt 5+ level LLMs for several years potentially. I’ve been wondering if that is actually partially why the propaganda wars or social media games have really escalated between nation states. In other words that sense of things been getting we…
Guess based on what? Soldiers are posting nuclear manuals to online flash card building tools. You think there'd be zero leaks about the existence of a GPT5 level thing? Let alone someone noticing that "hey, the NSA bought a lot of high end Nvidia GPUs lately, i wonder why".
I think they can be very successful in keeping most projects secret and yes the NSA has and does buy crazy amounts of hardware including GPUs and we even know they have various ASICs for a lot of things.
Occasionally there are leaks here and there you mention. But overall the secrecy rate is pretty high in my opinion.
This whole thing is like leaving 20 kids alone in a room with a marshmallow each and telling them don't eat it. ...and then expecting all of them resist. The debate around whether we should tell the kids not to eat it and for what reasons is completely academic. Practically this just isn't happening.
If they're really so worried about it they should start raiding data centres yesterday, writing open letters is such an obvious waste of time that I have a hard time taking it seriously.
I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.
I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…
Or maybe it would love us too hard and squish us. So even then we might be screwed!
It seems as though Scott just rejects the idea of the singularity entirely. If an AI gets advanced enough to improve itself, it seem entirely reasonable that it would go from laughable to godlike in a week. I don’t know if the singularity is near, inevitable at some point, or really even possible, but it does seem like something that at least could happen. And if it occurs, it will look exactly as what he describes n…
I see this stated a lot these days but it's not true.
You're imagining a exponential phenomenon, improvement leads to greater improvement, leading to still greater improvement, etc.
However, all exponential phenomenon require the right environment to sustain and, by their nature, consume that environment. Thus, they are inherently limited in scope and duration.
The bacteria in the Petri dish grows an exponential rate... until it reaches the edge of the dish and consumes the nutrients carefully places on it. The dynamite explodes for an instant and then stops exploding once the nitroglycerin is consumed.
Also, this is an especially unconcerning scenario because (1) we haven't seen step 1 of this process yet; (2) there's no particular reason to believe the environment necessary environment to sustain the exponential growth of AI is in place (if it is, it's random chance, and therefore very likely to fizzle out almost immediately).
This is a fine sci-fi scenario, but doesn't make sense in real life.
Earlier quoted context omitted.
I'm a lot more scared of an AI destroying humanity, like, every single person, than I am of any government or anything like that. More so than climate change. I'm not saying the people using it aren't dangerous -- but I would choose a totalitarian regime over a rogue AI any day of the week. It wouldn't be HAL 9000 or terminators. It would be an AI deciding that it needs to turn every bit of available matter on earth…
ChatGPT can do absolutely none of those things. Neither can any LLM. It’s not what they’re designed to do. There’s no need to worry about a speculative paper clip maximizer turning the world into grey goo. That’s still science fiction. The real harms today are much more banal.
Earlier quoted context omitted.
I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…
> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…
Earlier quoted context omitted.
> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…
This is known as the orthogonality thesis -- goals are orthogonal to intelligence. Intelligence is the ability to plan and act to achieve your goals, whatever they are. A stupid person can have a goal of helping others, and so can the smartest person on earth -- it's just that one is better. Likewise, a stupid person can have a goal of becoming wealthy, and so can a smart person. The smart person is Jeff Bezos or Bil…
Earlier quoted context omitted.
There are actually no relevant experts in the field of artificial general intelligence, or the safety thereof. No one has defined a clear path to build such a thing. Claiming to be an expert in this field is like claiming to be an expert in warp drives or time machines. Those calling for a halt in research are merely ignorant, or attention seeking grifters. Their statements can be dismissed out of hand regardless of…
You clearly have not read anything regarding the capabilities of GPT-4 and also clearly have not played around with ChatGPT at all. GPT-4 has already displayed a multitude of emergent capabilities. This is incredibly ignorant and I expect better from this forum.