Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

491–500 of 1001 posts

Re: Safe Superintelligence Inc.

#491

Post by schindlabua »

OtherShrezzing wrote: post #464
zeknife wrote: post #448

Earlier quoted context omitted.

Anything except tasks that require having direct control of a physical body. Until fully functional androids are developed, there is a lot a human-level AI can't do.

I think there's usually a difference between human-level and super-intelligent in these conversations. You can reasonably assume (some day) a superintelligence is going to 1) understand how to improve itself & undertake novel research 2) understand how to deceive humans 3) understand how to undermine digital environments If an entity with these three traits were sufficiently motivated, they could pose a material risk…

Deceiving a single human is pretty easy, but decieving the human super-organism is going to be hard.

Also, I don't believe in a singularity event where AI improves itself to godlike power. What's more likely is that the intelligence will plateau--I mean no software I have ever written effortlessly scaled from n=10 to n=10.000, and also humans understand how to improve themselves but they can't go beyond a certain threshold.

Re: Safe Superintelligence Inc.

#492

Post by JumpCrisscross »

Sharlin wrote: post #441
kjkjadksj wrote: post #285

Ilya's issue isn't developing a Safe AI. Its developing a Safe Business. You can make a safe AI today, but what happens when the next person is managing things? Are they so kindhearted, or are they cold and calculated like the management of many harmful industries today? If you solve the issue of Safe Business and eliminate the incentive structures that lead to 'unsafe' business, you basically obviate a lot of the so…

> You can make a safe AI today, but what happens when the next person is managing things? The point of safe superintelligence , and presumably the goal of SSI Inc., is that there won't be a next (biological) person managing things afterwards. At least none who could do anything to build a competing unsafe SAI. We're not talking about the banal definition of "safety" here. If the first superintelligence has any reason…

> there won't be a next (biological) person managing things afterwards. At least none who could do anything to build a competing unsafe SAI

This pitch has Biblical/Evangelical resonance, in case anyone wants to try that fundraising route [1]. ("I'm just running things until the Good Guy takes over" is almost a monarchic trope.)

[1] https://biblehub.com/1_corinthians/15-24.htm

Re: Safe Superintelligence Inc.

#493

Post by richie-guix »

esafak wrote: post #42
eigenvalue wrote: post #15

Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…

I think they will easily find enough capable altruistic people for this mission.

Not sure if spelling mistake.

Re: Safe Superintelligence Inc.

#494

Post by ohcmon »

chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

> glorified token predicting machine trained on existing data (made by humans)

sorry to disappoint, but human brain fits the same definition

Re: Safe Superintelligence Inc.

#495

Post by Nasrudith »

exe34 wrote: post #185
cherioo wrote: post #151

Earlier quoted context omitted.

I like your idea. But on the other hand, training an AGI, and then having a layer on top “aligning” the AGI sounds super dystopian and good plot for a movie.

the aligning means it should do what the board of directors wants, not what's good for society.

Poisoning Socrates was done because it was "good for society". I'm frankly even more suspicious of "good for society" than the average untrustworthy board of directors.

Re: Safe Superintelligence Inc.

#496

Post by nickdothutton »

One thing that strikes me about this time around the AI cycle, being old enough to have seen the late 80s, is how pessimistic and fearful society is as a whole now. Before… the challenge was too great, the investment in capital too draining, the results too marginal when compared to human labour or even “non-AI” computing.

I wonder if someone older still can comment on how “the atom” went from terrible weapon on war to “energy too cheap to meter” to wherever it is now (still a bête noire for the green energy enthusiasts).

Feels like we are over-applying the precautionary principle, the mainstream population seeing potential disaster everywhere.

Re: Safe Superintelligence Inc.

#497

Post by ToucanLoucan »

Nasrudith wrote: post #485
ToucanLoucan wrote: post #412

Earlier quoted context omitted.

I mean if the last 20 years is to be taken as evidence, it seems big tech is more than happy to shotgun unproven and unstudied technology straight into the brains of our most vulnerable populations and just see what the fuck happens. Results so far include a lot of benign nothing but also a whole lot of eating disorders, maxed out parents credit cards, attention issues, rampant misogyny among young boys, etc. Which,…

Please look up the history of maxing out credit cards, eating disorders, attention disorders, and misogyny. You seem to be under the mistaken impression that anything before your birth was the Garden of Eden and that the parade of horribles existed only because of "big tech". What is next? Blaming big tech for making teenagers horny and defiant?

> You seem to be under the mistaken impression that anything before your birth was the Garden of Eden and that the parade of horribles existed only because of "big tech"

Please point out where I said that. Because what I wrote was:

> I mean if the last 20 years is to be taken as evidence, it seems big tech is more than happy to shotgun unproven and unstudied technology straight into the brains of our most vulnerable populations and just see what the fuck happens. Results so far include a lot of benign nothing but also a whole lot of eating disorders, maxed out parents credit cards, attention issues, rampant misogyny among young boys, etc. Which, granted, the readiness to fuck with populations at scale and do immeasurable harm doesn't really make tech unique as an industry, just more of the same really.

Which not only is not romanticizing the past, in fact I directly point out that making tons of people's lives worse for profit was a thing in industry long before tech came along, but also do not directly implicate tech as creating sexism, exploiting people financially, or fucking up young women's brains any differently, simply doing it more. Like most things with tech, it wasn't revolutionary new social harms, it was just social harms delivered algorithmically, to the most vulnerable, and highly personalized to what they are acutely vulnerable to in specific.

That is not a new thing, by any means, it's simply better targeted and more profitable, which is great innovation providing you lack a conscience and see people as only a resource to be exploited for your own profit, which a lot of the tech sector seems to.

Re: Safe Superintelligence Inc.

#498

Post by bbor »

chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

Well, an entire industry of researchers, which used to be divided, is now uniting around calls to slow development and emphasize safety (like, “dissolve companies” emphasis not “write employee handbooks” emphasis). They’re saying, more-or-less in unison, that GPT3 was an unexpected breakthrough in the Frame Problem, based on Judea Pearl’s prescient predictions. If we agree on that, there are two options:

1. They’ve all been tricked/bribed by Sam Altman and company (which btw this is a company started against those specific guys, just for clarity). Including me, of course.

2. You’re not as much of an expert in cognitive science as you think you are, and maybe the scientists know something you don’t.

With love. As much love as possible, in a singular era

Re: Safe Superintelligence Inc.

#499

Post by RGBCube »

The year is 2022. An OpenAI employee concered about AI safety creates his own company.

The year is 2023. An OpenAI employee concered about AI safety creates his own company.

The year is 2024.

Re: Safe Superintelligence Inc.

#500

Post by foolishbard »

chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

There's a chance that these systems can actually out perform their training data and be better than the sum of their parts. New work out Harvard talks about this idea of "transcendence" https://arxiv.org/abs/2406.11741

While this is a new area, it would be naive to write this off as just science fiction.

Post reply on HN