Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

711–720 of 1001 posts

Re: Safe Superintelligence Inc.

#711

Earlier quoted context omitted.

This. Every nation-state will be in the game. Private enterprise will be in the game. Bitcoin-funded individuals will be in the game. Criminal enterprises will be in the game. How does one company building a safe version stop that? If I have access to hardware and data how does a safety layer get enforced? Regulations are for organizations that care about public perception, the law, and stock prices. Criminals and na…

I think the honest view (and you can scoff at it) is that winning the SI race basically wins you the enforcement race for free

That's why it's called an arms race, and it does not really end in this predictable manner.

The party that's about to lose will use any extrajudicial means to reclaim their victory, regardless of the consequences, because their own destruction would be imminent otherwise. This ultimately leads to violence.

Re: Safe Superintelligence Inc.

#712
post #8

This makes sense. Ilya can probably raise practically unlimited money on his name alone at this point. I'm not sure I agree with the "no product until we succeed" direction. I think real world feedback from deployed products is going to be important in developing superintelligence. I doubt that it will drop out of the blue from an ivory tower. But I could be wrong. I definitely agree that superintelligence is within…

His idea that only corporations and governments should have access to this product. He doesn’t think people should have access even to ChatGPT or LLMs. Goal is to build companies with evaluations of dozens, hundreds trillions of dollars and make sure only US government will have access to super intelligence to surpass other countries economy and military wise, ideally to solidify US hegemony and undermine other countries economies and progress towards super intelligence.

I mean who wouldn’t trust capitalists that are laying of people by thousands just to please investors or government that is “under-intelligent” and hasn’t brought anything but pain and suffering to other countries.

Re: Safe Superintelligence Inc.

#713

Rabotai, Hatzlacha. If you fail, the human world will go down in flames. But, no pressure! Anyway, you said a "cracked" team, whereas the usual expression for elite excellence is "crack". "Cracked" on the other hand means: Insane. Well.... perhaps what we need here is some of both! Looking forward to your success... indeed, it is our only hope.

"cracked" is some gamer lingo for really good. I'm assuming they're referring to that.

https://www.urbandictionary.com/define.php?term=Cracked

Re: Safe Superintelligence Inc.

#714
post #712
post #8

This makes sense. Ilya can probably raise practically unlimited money on his name alone at this point. I'm not sure I agree with the "no product until we succeed" direction. I think real world feedback from deployed products is going to be important in developing superintelligence. I doubt that it will drop out of the blue from an ivory tower. But I could be wrong. I definitely agree that superintelligence is within…

His idea that only corporations and governments should have access to this product. He doesn’t think people should have access even to ChatGPT or LLMs. Goal is to build companies with evaluations of dozens, hundreds trillions of dollars and make sure only US government will have access to super intelligence to surpass other countries economy and military wise, ideally to solidify US hegemony and undermine other count…

Personally I wouldn’t trust OpenAi to work on super intelligence - it can indeed cause mass extinction. Government is completely different story they will specifically train AI to develop biological, chemical and weapons of mass destruction. Train it to strategize and plan on how to win war conflicts, social engineering and manipulations, hacking. And obviously will let it control drone planes and tanks, artillery. Give it access to satellites and so on. Nothing can go wrong when jarheads are at work :). Maybe it will even find the trillions of dollars that Pentagon can’t find during every audit they can’t pass.

Re: Safe Superintelligence Inc.

#716
post #712
post #8

This makes sense. Ilya can probably raise practically unlimited money on his name alone at this point. I'm not sure I agree with the "no product until we succeed" direction. I think real world feedback from deployed products is going to be important in developing superintelligence. I doubt that it will drop out of the blue from an ivory tower. But I could be wrong. I definitely agree that superintelligence is within…

His idea that only corporations and governments should have access to this product. He doesn’t think people should have access even to ChatGPT or LLMs. Goal is to build companies with evaluations of dozens, hundreds trillions of dollars and make sure only US government will have access to super intelligence to surpass other countries economy and military wise, ideally to solidify US hegemony and undermine other count…

And obviously one of the points that people don’t need AI tools, because corporations need agent like AIs that can quickly replace all the staff.

Re: Safe Superintelligence Inc.

#717
post #498

Earlier quoted context omitted.

Well, an entire industry of researchers, which used to be divided, is now uniting around calls to slow development and emphasize safety (like, “dissolve companies” emphasis not “write employee handbooks” emphasis). They’re saying, more-or-less in unison, that GPT3 was an unexpected breakthrough in the Frame Problem, based on Judea Pearl’s prescient predictions. If we agree on that, there are two options: 1. They’ve a…

Are they actually united? Or is this the ai safety subfaction circling the wagons due to waning relevance in the face of not-actually-all-that-threatening ai?

I personally find that summary of things to be way off the mark (for example, hopefully "the face" you reference isn't based on anything that appears in a browser window or in an ensemble of less than 100 agents!) but I'll try to speak to the "united" question instead.

1. The "Future of Life" institute is composed of lots of very serious people who recently helped get the EU "AI Act" passed this March, and they discuss the "myriad risks and harms AI presents" and "possibly catastrophic risks". https://newsletter.futureoflife.org/p/fli-newsletter-march-2...

2. Many researchers are leaving large tech companies, voicing concerns about safety and the downplaying of risks in the name of moving fast and beating vaguely-posited competitors. Both big ones like Hinton and many, many smaller ones. I'm a little lazy to scrape the data together, but it's such a wide phenomenon that a quick Google/Kagi should be enough for a vague idea. This is why Anthropic was started, why Altman was fired, why Microsoft gutted their AI safety org, and why Google fired the head of their AI ethics team. We forgot about that one cause it's from before GPT3, but it doesn't get much clearer than this:

> She co-authored a research paper which she says she was asked to retract. The paper had pinpointed flaws in AI language technology, including a system built by Google... Dr Gebru had emailed her management laying out some key conditions for removing her name from the paper, and if they were not met, she would "work on a last date" for her employment. According to Dr Gebru, Google replied: "We respect your decision to leave Google... and we are accepting your resignation."

3. One funny way to see this happening is to go back to seminal papers from the last decade and see where everyone's working now. Spoiler alert: not a lot of the same names left at OpenAI, or Anthropic for that matter! This is the most egregious I've found -- the RLHF paper: see https://arxiv.org/pdf/2203.02155

3. Polling of AI researchers shows a clear and overhelming trend towards AGI timelines being moved up significantly. It's still a question deeply wrapped up in accidental factors like religious belief, philosophical perspective, and general valence as a person, so I think the sudden shift here should tell you a lot. https://research.aimultiple.com/artificial-general-intellige...

The article I just linked actually has a section where they collect caveats, and the first is this Herbert Simon quote from 1965 that clearly didn't age well: "Machines will be capable, within twenty years, of doing any work a man can do.” This is a perfect example of my overall point! He was right. The symbolists were right, are right, will always be right -- they just failed to consider that the connectionists were just as right. The exact thing that stopped his prediction was the frame problem, which is what we've now solved.

Hopefully that makes it a bit clearer why I'm anxious all the time :). The End Is Near, folks... or at least the people telling you that it's definitely not here have capitalist motivations, too. If you count the amount of money lost and received by each "side" in this "debate", I think it's clear the researcher side is down many millions in lost salaries and money spent on thinktank papers and Silicon Valley polycule dorms (it's part of it, don't ask), and the executive side is up... well, everything, so far. Did you know the biggest privately-funded infrastructure project in the history of humanity was announced this year? https://www.datacenterdynamics.com/en/opinions/how-microsoft...

Re: Safe Superintelligence Inc.

#718

Ilya Sutskever I recognize from OpenAI, Daniel Gross I have met several times (friend of multiple friends), Daniel Levy I do not recognize at all. Who is he?

I Googled for: "Daniel Levy ai"

First hit: https://ai.stanford.edu › ~danilevy Bio. I currently lead the Optimization team at OpenAI. Before that, I was a Ph.D. student in Computer Science at Stanford University advised by John Duchi.>>

Re: Safe Superintelligence Inc.

#719

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

The size of the gap between “smarter than humans” and “not controlled by humans anymore” is obviously where the disagreement is. To assume it’s a chasm that can never be overcome, you need at least the following to be true: No amount of focus or time or intelligence or mistakes in coding will ever bridge the gap. That rules and safeguards can be made that are perfectly inescapable. And nobody else will get enough pow…

I think we should assume it will be badly aligned. Not only are there the usual bugs and unforeseen edge conditions, but there are sure to be unintended consequences. We have a long, public history of unintended consequences in laws, which are at least publicly debated and discussed. But perhaps the biggest problem is that computers are, by nature, unthinking bureaucrats who can't make the slightest deviation from the rules no matter how obviously the current situation requires it. This makes people livid in a hurry. As a non-AI example (or perhaps AI-anticipating), consider Google's customer support...

Re: Safe Superintelligence Inc.

#720
post #691

Earlier quoted context omitted.

I think the common line of thinking here is that it won't be actively antagonist to , rather it will have goals that are orthogonal to ours. Since it is superintelligent, and we are not, it will achieve its goals and we will not be able to achieve ours. This is a big deal because a lot of our goals maintain the overall homeostasis of our species, which is delicate! If this doesn't make sense, here is an ungrounded, n…

It builds stuff? First they would have to do that over our dead bodies which means they already somehow able to build stuff without competing with us for resources, it’s a chicken or the egg problem you see?

Valid call out, this was firmly motivated by it being a super intelligence.
Post reply on HN