Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

701–710 of 1001 posts

Re: Safe Superintelligence Inc.

#701

Post by bnralt »

whimsicalism wrote: post #643
ToucanLoucan wrote: post #503

Earlier quoted context omitted.

> Do you think the internet has returned us to pre-women's-suffrage levels of misogyny? Well in the States at least we did just revoke a sizable amount of their bodily autonomy so, the situation may not be that bad, yet, but I wouldn't call it good by any measurement. Any my objection isn't "that sexism exists in society," that is probably going to be true as a statement until the sun explodes, and possibly after tha…

I am firmly in favor of abortion rights but still I do not think that is even remotely a good bellwether to measure sexism/misogyny. 1. Women are more likely than men to be opposed to abortion rights. 2. Many people who are opposed to abortion rights have legitimately held moral concerns that are not simply because they have no respect for women's rights. 3. Roe v. Wade was the decision of 9 people. It absolutely did…

Right. We can see something similar with the Terry Schiavo case or the opposition to IFV right now. It's clear that this is about different opinions regarding what should be considered a living human being (which doesn't seem to have a very clean definition for anyone, it should be noted). Depending on where you draw the line, it's either horrible to outlaw abortions or horrible to allow them.

Framing it as simply taking away a woman's bodily autonomy is like framing circumcision as simply being about mutilating men.

Re: Safe Superintelligence Inc.

#702

Post by id00 »

hackerlight wrote: post #389
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

China can not win this race and I hate that this comment is going to be controversial among the circle of people that need to understand this the most. It is damn frightening that an authoritarian country is so close to number one in the race to the most powerful technology humanity has invented, and I resent people who push for open source AI for this reason alone. I don't want to live in a world where the first sup…

I agree with your point. However I also don't want to live in a world where the first superintelligence is controlled by an entities that:

- try to scan all my chat messages searching for CSAM

- have black sites across the world where anyone can dissappear without any justice

- can require me to unlock my phone and give it away

- ... and so on

The point I'm trying to make is that other big players in the race are crooked as well and i'm waiting for a great horror for AGI to be invented as no matter who gets it - we are all doomed

Re: Safe Superintelligence Inc.

#703

Post by richardw »

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

The size of the gap between “smarter than humans” and “not controlled by humans anymore” is obviously where the disagreement is.

To assume it’s a chasm that can never be overcome, you need at least the following to be true:

No amount of focus or time or intelligence or mistakes in coding will ever bridge the gap. That rules and safeguards can be made that are perfectly inescapable. And nobody else will get enough power to overcome our set of controls.

I’m less worried bad actors control it than I am that it escapes them and is badly aligned.

Re: Safe Superintelligence Inc.

#704

Post by theptip »

jen729w wrote: post #689
theptip wrote: post #632

Earlier quoted context omitted.

Trouble is, in practice what you would need to do might be “turn off all of Google’s datacenters”. Or perhaps the thing manages to secure compute in multiple clouds (which is what I’d do if I woke up as an entity running on a single DC with a big red power button on it). The blast radius of such decisions are large enough that this option is not trivial as you suggest.

Right, but when we’ve literally invented a superintelligence that we’re worried will eliminate the race, a) we’ve done that, which is cool. Now let’s figure out how to control it. b) you can’t get your gmail for a bit while we reboot the DC. That’s probably okay.

a) after you create the superintelligence is likely too late. You seem to think that inventing superintelligence means that we somehow understand what we created, but note that we have no idea how a simple LLM works, let alone an ASI that is presumably 5-10 OOM more complex. You are unlikely to be able to control a thing that is way smarter than you, the safest option is to steer the nature of that thing before it comes into being (or, don’t build it at all). Note that we currently don’t know how to do this, it’s what Ilya is working on. The approach from OpenAI is roughly to create ASI and then hope it’s friendly.

b) except that is not how these things go in the real world. What actually happens is that initially it’s just a risk of the agent going rogue, the CEO weighs the multi-billion dollar cost vs. some small-seeming probability of disaster and decides to keep the company running until the threat is extremely clear, which in many scenarios is too late.

(For a recent example, consider the point in the spread of Covid where a lockdown could have prevented the disease from spreading; likely somewhere around tens to hundreds of cases, well before the true risk was quantified, and therefore drastic action was not justified to those that could have pressed the metaphorical red button).

Re: Safe Superintelligence Inc.

#705

Post by jpk »

jrflowers wrote: post #680
akira2501 wrote: post #436

Earlier quoted context omitted.

> for society and young people (girls in particular [1]). I don't think the article with a single focused example bears that out at all. From the article: > "Even more troubling are the men who signed up for paid subscriptions after the girl launched a program for super-fans receive special photos and other content." > "Her mom conceded that those followers are “probably the scariest ones of all.”" I'm sorry.. but wh…

> I'm sorry.. but what is your daughter selling, exactly? Did.. did you just post “I’d have to see these images of the preteen girl before I could form an opinion about whether or not the men buying and sharing them were creeps” as a rebuttal to an article about social media enabling child predators?

Since nobody bothered to explain their downvotes, I'll say it: This is a pretty uncharitable reading of the quoted comment.

Re: Safe Superintelligence Inc.

#706

Post by akira2501 »

jrflowers wrote: post #680
akira2501 wrote: post #436

Earlier quoted context omitted.

> for society and young people (girls in particular [1]). I don't think the article with a single focused example bears that out at all. From the article: > "Even more troubling are the men who signed up for paid subscriptions after the girl launched a program for super-fans receive special photos and other content." > "Her mom conceded that those followers are “probably the scariest ones of all.”" I'm sorry.. but wh…

> I'm sorry.. but what is your daughter selling, exactly? Did.. did you just post “I’d have to see these images of the preteen girl before I could form an opinion about whether or not the men buying and sharing them were creeps” as a rebuttal to an article about social media enabling child predators?

Well, I already have an opinion on them. The article seems to purposefully avoid describing what the market and the service here is. There are a limited number of venues where a preteen girl can credibly earn money on social media. Also, if she's earning that money, I'd openly wonder whether a Coogan Account is appropriate and if one exists here.

Anyways.. my shock was really over the fact that the mother is basically saying "I want to sell access to my daughter online but I'm surprised that the biggest spenders are adult men with questionable intentions." Did she genuinely believe that the target market was other 12 year old girls willing to pay to watch another 12 year old girl? The parents resignation over the situation in deference to the money is also disgusting.

> about social media enabling child predators?

That's my point. The article entirely fails to do that. It's one case with a questionable background and zero investigation over the claim, which you'd expect, because the crime statistics show the exact opposite.

Re: Safe Superintelligence Inc.

#707

Post by yonatron »

Rabotai, Hatzlacha. If you fail, the human world will go down in flames. But, no pressure! Anyway, you said a "cracked" team, whereas the usual expression for elite excellence is "crack". "Cracked" on the other hand means: Insane. Well.... perhaps what we need here is some of both! Looking forward to your success... indeed, it is our only hope.

Re: Safe Superintelligence Inc.

#708

Post by hackerlight »

richrichie wrote: post #700
hackerlight wrote: post #389

Earlier quoted context omitted.

China can not win this race and I hate that this comment is going to be controversial among the circle of people that need to understand this the most. It is damn frightening that an authoritarian country is so close to number one in the race to the most powerful technology humanity has invented, and I resent people who push for open source AI for this reason alone. I don't want to live in a world where the first sup…

US is no angel and it cannot be the only one which wins the race. We have hard evidence of how monopoly power gets abused in the case of the US e.g. as the sole nuclear power, it used nukes on civilians. We need every one to win this race to keep things on balance.

US has to win the race because while it's true that it's no angel, it isn't an authoritarian dictatorship and there isn't an equivalence in how bad the world will end up for you and me if the authoritarian side wins the race. Monopoly power will get abused the most by the least democratic actors, which is China. We need multiple actors within the US to win to balance power. We don't need or want China to be one of the winners. There is no upside for humanity in that outcome.

The US policymakers have figured this out with their chip export ban. Techies on the other hand, probably more than half the people here, are so naive and clueless about the reality of the moment we are in, that they support open sourcing this tech, the opposite of what we need to be doing to secure our future prosperity and freedom. Open source almost anything, just not this. It gives too much future power to authoritarians. That risk overwhelms the smaller risks that open sourcing is supposed to alleviate.

Re: Safe Superintelligence Inc.

#709

Post by bbor »

nradov wrote: post #631
bbor wrote: post #498

Earlier quoted context omitted.

Well, an entire industry of researchers, which used to be divided, is now uniting around calls to slow development and emphasize safety (like, “dissolve companies” emphasis not “write employee handbooks” emphasis). They’re saying, more-or-less in unison, that GPT3 was an unexpected breakthrough in the Frame Problem, based on Judea Pearl’s prescient predictions. If we agree on that, there are two options: 1. They’ve a…

We don't agree on that. They're just making things up with no real scientific evidence. There are way more than 2 options.

What kind of real scientific evidence are you looking for? What hypotheses have they failed to test? To the extent that we're discussing a specific idea in the first place ("are we in a qualitatively new era of AI?" perhaps), I'm struggling to imagine what your comment is referencing.

You're of course right that there are more than two options in an absolute sense, I should probably limit the rhetorical flourishes for HN! My argument is that those are the only supportable narratives that answer all known evidence, but it is just an argument.

Re: Safe Superintelligence Inc.

#710

Post by zucker42 »

m3kw9 wrote: post #455
tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

Why do people always think that a superintelligent being will always be destructive/evil to US? I rather have the opposite view where if you are really intelligent, you don’t see things as a zero sum game

Convergent instrumental goals[1] and the orthogonality thesis[2], among other reasons.

[1] https://youtu.be/ZeecOKBus3Q?si=cYJUaxjIJPIbubRL

[2] https://youtu.be/hEUO6pjwFOo?si=DXVosLh6YTsMkKOx

Post reply on HN