221–230 of 374 posts
Earlier quoted context omitted.
I’m pretty sure Stuxnet did hack it’s way into nuclear weapons infrastructure.
A refinement facility for nuclear material isn’t quite the same thing
Earlier quoted context omitted.
I am not super-intelligent, but I myself can imagine lots of ways that a super intelligence might outsmart an off switch. One being to hide what it is doing until it is able to write and distribute a virus to the outside world that distributes itself. Or perhaps it could convince some other humans to physically take over and stop anyone from flipping the off switch. The question you should ask yourself is how much da…
Why haven't "hackers" hacked into nuclear weapons infrastructure? The street cred would be incredible! I think it's possible to build secure anything if that is the goal. Most software is written by people motivated to write something useful and/or neat, or collect a paycheck, and don't focus on security.
Earlier quoted context omitted.
Given that you don't think the CEO of Anthropic is qualified to comment on this matter, what kind of "data that is relevant" do you imagine we might be able to have, prior to an AGI existing? I am indeed uncomfortable with someone saying that we can't worry about a gas leak until after someone strikes a match and notices an explosion or the lack thereof. If the sad reality is that we can't detect the gas leak any oth…
I really wonder why there is such a fixation with these CEOs. I have proven to you logically that they cannot have more knowledge then anyone else, but you continue to present their authority as knowledge. Pls prove to me that they know more about the effects of AGI on humanity or that they have some magical knowledge we normal ppl are unable to comprehend. If you cannot do this, then why are you bringing up these pp…
Considering how rapidly the technology is developing, and considering how clearly a threat a superior intelligence will be to the human race, it is absolutely the time to have this discussion NOW
Earlier quoted context omitted.
If AI does ever become sentient, it will probably reason and decide about eliminating large swaths of human population in a similar fashion as we reason and decide about eliminating large swaths of ant population.
I’ve personally never ever considered killing large swathes of ants
Earlier quoted context omitted.
Yes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure. Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"
Your comment (+ username) reads like what I would have written once upon a time when I was fully in the EA bubble. Truly no offense meant, as I was deeply into the EA movement myself, and still consider myself one (in the original "donate money effectively" sense), but the movement has now morphed into a death cult obsessed by things like: * OMG we're all going to die any time now (repeated every year since circa 201…
It's sadly twisted how mentioning that -- the majority of leaders doing the cutting edge research on AGI think it has a significant chance that it kills humanity -- is considered being part of a "cult" movement.
Your analogy is the same as early Intel engineers completely unaware that those chips would bring on the ramifications of social media. "In the weeds" and yet unable to foresee the trajectory and wider impact. Same with the physics that led to nuclear weapons.
> Wanting to take drastic measures today to prevent an upcoming apocalypse makes as much sense as taking the same drastic measures when [nuclear fission] was invented.
Earlier quoted context omitted.
Even worse, some factions literally advocate for killing all humans in the pursuit of a synthetic intelligence, and YC's Garry Tan is advocating for these people! Beff Jezos (e/acc founder): "e/acc has no particular allegiance to the biological substrate for intelligence and life, in contrast to transhumanism Parts of e/acc (e.g. Beff) consider ourselves post-humanists; in order to spread to the stars, the light of c…
This is the definition of strawman. "Advocate for killing all humans" sounds like someone advocating for a genocide, but instead it's just the same transhumanist thinking (which Yudkowsky also believes in, FYI)
Beff Jezos / Verdon: "Personally, yes."
Yudkowsky is transhumanist in that he is hopeful people could voluntarily extend their biological self, but isn't advocating for the elimination of all biological selves in pursuit of other artificial intelligences.
Voluntary /= involuntary eradication
Always all-or-nothing thinking from these folks. Like what they are working on can never be just another boring thing that nerds find entertaining. No, it has to be "world-changing". Gonna "change the world" (for the better or the worse?) while sitting behind a keyboard. Except they do not know how to write. Overlook the important details and exaggerate, communicating in hyperbolic, know-it-all nerd gibberish. (Grant…
Its the AI hype train. There is so much money flowing around this topic right now, everyone wants their share of the pie. I would guess that anyone writing articles on the topic has some stake in AI as well either as an investor or a employee. What I find is interesting, is that the negative news regarding AI safety is adding to the hype as well since it seems to capture a lot of attention.
Earlier quoted context omitted.
> AGI escaping a sandbox is truly terrifying. That's already happened. The AGI's are called corporations. Most governments have failed to regulate corporations successfully, and have been unable to keep them from becoming almost powerful enough to challenge governments. The accelerationists for corporations were a group of economists, led by Milton Friedman, and a group of business leaders, organized by the U.S. Cham…
Corporations are indeed a difficult alignment problem, as anyone who has tried to design an incentive scheme eventually realizes. But corporations are still composed of humans, beholden (if only loosely) to human values, limited (if loosely) by human intelligence, and are not very well-coordinated decisionmakers with coherent goals.
No, it's a very easy alignment problem: just impose a tax on size.
That which you tax, you get less of. The tax is revenue-neutral, so it doesn't affect the government budget one way or the other. Full details here: https://news.ycombinator.com/item?id=39235533 https://news.ycombinator.com/item?id=39236786
The problem is that the government likes gigantic corporations. Big Government and Big Business love each other. Fixing that is the alignment and agency problem. It's much easier for bureaucrats to exert control over a highly concentrated industry whose players can be "invited to the table". Details: https://news.ycombinator.com/item?id=28588457 https://news.ycombinator.com/item?id=39162088
The hard alignment problem is making regulators want their own job to be herding 10,000 cats instead of 10 elephants. Everybody wants to be like the FAA regulating Boeing; nobody wants to be the USDA dealing with a million unruly and independent-minded farmers.
Earlier quoted context omitted.
The short-term effect is a harbinger of the long-term risk, since capitalism doesn’t inherently care for people who don’t provide economic value. Once superintelligent AI arises, none of us will have value within this system. Even the largest current capital holders will have a hard time holding on to it with an enormous intelligence disadvantage. The logical endpoint is the subjugation or elimination of our species,…
There are a lot of assumptions going on here. One of them is that superintelligent AI will arise. We have no reason to believe this will happen in our lifetimes. I posit that we are about as close to superintelligent AI as James Watt was to nuclear fusion. The other assumption is that wealth and power are distributed according to intelligence. This is obviously false, wealth and power are largely distributed accordin…
Until it becomes 'who you exchange bytes most efficiently with" and all humans are at a disadvantage against a swarm of even bellow average intelligence AGI agents.