Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

761–770 of 1001 posts

Re: Safe Superintelligence Inc.

#761

Post by bottlepalm »

Xenoamorphous wrote: post #471
tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

I wonder if this is an Ian Malcolm in Jurassic Park situation, i.e. “your scientists were so preoccupied with whether they could they didn t stop to think if they should”. Maybe the only way to avoid an unsafe superintelligence is to not create a superintelligence at all.

It’s exactly that. You’re a kid with a gun creating dinosaurs all cavalier. And a fool to think you can control them.

Re: Safe Superintelligence Inc.

#762

Post by calf »

deathanatos wrote: post #755

I just can't read something about "safe superintelligence" and not hear the opening plot to Friendship is Optimal , a short story about My Little Pony ( yes , you read that right), virtual worlds, and the singularity; it's a good read¹. One of the characters decides they can't make a violent video game, as the AI would end up violent. Better to build a game like MLP, so that the AI isn't going to destroy humanity, or…

To me, "safe superintelligence" presumes it is practically feasible and morally justifiable to create such a thing. I don't share those presuppositions.

Re: Safe Superintelligence Inc.

#763

Post by creer »

gavin_gee wrote: post #232
insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

This. Every nation-state will be in the game. Private enterprise will be in the game. Bitcoin-funded individuals will be in the game. Criminal enterprises will be in the game. How does one company building a safe version stop that? If I have access to hardware and data how does a safety layer get enforced? Regulations are for organizations that care about public perception, the law, and stock prices. Criminals and na…

"nation state" doesn't mean what you think it means.

More constructively, I don't know that very much will stop even a hacker from getting time on the local corporate or university AI and get it to do some "work". After all the first thing the other kind of hacker tried with generative AI is to get them to break out of their artificial boundaries, and hook them to internet resources. I don't know that anyone has hooked up a wallet to one yet - but I have no doubt that people have tried. It will be fun.

Re: Safe Superintelligence Inc.

#764

Post by jrflowers »

jpk wrote: post #705
jrflowers wrote: post #680

Earlier quoted context omitted.

> I'm sorry.. but what is your daughter selling, exactly? Did.. did you just post “I’d have to see these images of the preteen girl before I could form an opinion about whether or not the men buying and sharing them were creeps” as a rebuttal to an article about social media enabling child predators?

Since nobody bothered to explain their downvotes, I'll say it: This is a pretty uncharitable reading of the quoted comment.

This is a good point. When you read this:

> Instagram’s algorithms have steered men with deviant sexual attraction to children to the girl’s page, flooding it with unwanted comments and come-ons, according the Journal.

It is a normal response to think “I wonder what was so attractive about the images that the child made that these men were forced to follow and comment on them. I will post about how we shouldn’t judge the men or platform until we’ve seen the images”

Re: Safe Superintelligence Inc.

#765

Post by fromMars »

null0pointer wrote: post #528
fromMars wrote: post #268

Earlier quoted context omitted.

I find this hard to believe having worked in multiple enterprises and in the FAANG world. In my anecdotal experience, I can only think of one or two examples of someone from the enterprise world who I would consider outstanding. The overall quality of engineers is much higher at the FAANG companies.

I have also worked in multiple different sized companies, including FAANG, and multiple countries. My assessment is that FAANGs tend to select for generally intelligent people who can learn quickly and adapt to new situations easily but who nowadays tend to be passionless and indifferent to anything but money and prestige. Personally I think passion is the differentiator here, rather than talent, when it comes to doi…

I sort of agree and disagree. I wouldn't agree with the idea that most FAANG engineers are not passionate by nature about their work.

What I would say is that the bureaucracy and bullshit one has to deal with makes it hard to maintain that passion and that many end up as TC optimizers in the sense that they stay instead of working someplace better for less TC.

That said, I am not sure how many would make different choices. Many who join a FAANG company don't have the slightest inkling of what it will be like and once they realize that they are tiny cog in a giant machine it's hard to leave the TC and perks behind.

Re: Safe Superintelligence Inc.

#766

Post by botanical »

throwaway4good wrote: post #735
rqtwteye wrote: post #730

I always wonder about "safe" for who? If the current economic system continues, we may end up with a lot of people out of jobs. Add to that improvements in robotics and we will have many people ending up having nothing to contribute to the economy. I am not getting the impression that the people who push for AI safety are thinking about this. It seems they are most worried about not losing their position of privilege…

It is a strange thing. We are not anywhere near “super intelligence” yet the priority is safety. If the Wright brothers had focused on safety, I am not sure they would have flown very far.

The consequences of actual AI is far reaching compared to a plane crash.

Re: Safe Superintelligence Inc.

#767

Post by mr_toad »

comfysocks wrote: post #747
rqtwteye wrote: post #730

I always wonder about "safe" for who? If the current economic system continues, we may end up with a lot of people out of jobs. Add to that improvements in robotics and we will have many people ending up having nothing to contribute to the economy. I am not getting the impression that the people who push for AI safety are thinking about this. It seems they are most worried about not losing their position of privilege…

The Industrial Revolution devalued manual labor. Sure, new jobs were created, but on the whole this looked like a shift to knowledge work and away from manual labor. Now AI is beginning to devalue knowledge work. Although the limits of current technology is obvious in many cases, AI is already doing a pretty good job at replacing illustrators and copy writers. It will only get better. Who owns the value created by AI…

> The Industrial Revolution devalued manual labor.

Only some types. It caused a great number of people to be employed in manual labour, in the new factories. The shift to knowledge work came much later as factory work (and farming) became increasingly automated.

Re: Safe Superintelligence Inc.

#768

Post by Atotalnoob »

frenchie4111 wrote: post #14

I am not on the bleeding edge of this stuff. I wonder though: How could a safe super intelligence out compete an unrestricted one? Assuming another company exists (maybe OpenAI) that is tackling the same goal without spending the cycles on safety, what chance do they have to compete?

Safety techniques require you to understand your product and have deep observability.

This and safety techniques themselves can improve the performance of the hypothetical AGI.

RLHF was originally an alignment tool, but it improves llms significantly

Re: Safe Superintelligence Inc.

#769

Post by jazzyjackson »

johnthewise wrote: post #549
ilrwbwrkhv wrote: post #224

Earlier quoted context omitted.

There is no "superintelligence" or "AGI". People are falling for marketing gimmicks. These models will remain in the word vector similarity phase forever. Till the time we understand consciousness, we will not crack AGI and then it won't take brute forcing of large swaths of data, but tiny amounts. So there is nothing to worry. These "apps" might be as popular as Excel, but will go no further.

If you described Chatgpt to me 10 years ago, I would have said it's AGI.

I don't feel that much has changed in the past 10 years. I would have done the same thing then as now, spent a month captivated by the crystal ball until I realized it was just refracting my words back at me.

Re: Safe Superintelligence Inc.

#770

Post by jazzyjackson »

hackerlight wrote: post #708
richrichie wrote: post #700

Earlier quoted context omitted.

US is no angel and it cannot be the only one which wins the race. We have hard evidence of how monopoly power gets abused in the case of the US e.g. as the sole nuclear power, it used nukes on civilians. We need every one to win this race to keep things on balance.

US has to win the race because while it's true that it's no angel, it isn't an authoritarian dictatorship and there isn't an equivalence in how bad the world will end up for you and me if the authoritarian side wins the race. Monopoly power will get abused the most by the least democratic actors, which is China. We need multiple actors within the US to win to balance power. We don't need or want China to be one of th…

Which countries will become authoritarian dictatorships on a 25 year timeline is not easily foreseeable.
Post reply on HN