Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

731–740 of 1001 posts

Re: Safe Superintelligence Inc.

#731

Post by cozzyd »

auggierose wrote: post #207
PheonixPharts wrote: post #89

Earlier quoted context omitted.

> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…

TC optimization being tail call optimization?

Curie temperature

Re: Safe Superintelligence Inc.

#733

Post by fernly »

fnordpiglet wrote: post #68

“””We are assembling a lean, cracked team of the world’s best engineers and researchers dedicated to focusing on SSI and nothing else.””” Cracked indeed

For fun I asked Claude (sonnet) to proofread that paragraph and it found no problem with "cracked team".

Re: Safe Superintelligence Inc.

#734

Post by _kb »

cherioo wrote: post #151
aresant wrote: post #39

Prediction - the business model becomes an external protocol - similar to SSL - that the litany of AI companies working to achieve AGI will leverage (or be regulated to use) From my hobbyist knowledge of LLMs and compute this is going to be a terrifically complicated problem, but barring a defined protocol & standard there's no hope that "safety" is going to be executed as a product layer given all the different appr…

I like your idea. But on the other hand, training an AGI, and then having a layer on top “aligning” the AGI sounds super dystopian and good plot for a movie.

We create superintelligence, but just feed it a steady dose of soma.

Re: Safe Superintelligence Inc.

#735

Post by throwaway4good »

rqtwteye wrote: post #730

I always wonder about "safe" for who? If the current economic system continues, we may end up with a lot of people out of jobs. Add to that improvements in robotics and we will have many people ending up having nothing to contribute to the economy. I am not getting the impression that the people who push for AI safety are thinking about this. It seems they are most worried about not losing their position of privilege…

It is a strange thing. We are not anywhere near “super intelligence” yet the priority is safety.

If the Wright brothers had focused on safety, I am not sure they would have flown very far.

Re: Safe Superintelligence Inc.

#736

Post by bbor »

eigenvalue wrote: post #15

Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…

Great analysis, but you're missing two key factors IMO:

1. People who honestly think AGI is here aren't thinking about their careers in the typical sense at all. It's sorta ethical/"ideological", but it's mostly just practical.

2. People who honestly think AGI is here are fucking terrified right now, and were already treating Ilya as a spiritual center after Altman's coup (quite possibly an unearned title, but oh well, that's history for ya). A rallying cry like this -- so clearly aimed at the big picture instead of marketing they don't even need CSS -- will be seen as a do-or-die moment by many, I think. There's only so much of "general industry continues to go in direction experts recommend against; corporate consolidation continues!" headlines an ethical engineer can take before snapping and trying to take on Goliath, odds be damned

Re: Safe Superintelligence Inc.

#737

Post by IncreasePosts »

rqtwteye wrote: post #730

I always wonder about "safe" for who? If the current economic system continues, we may end up with a lot of people out of jobs. Add to that improvements in robotics and we will have many people ending up having nothing to contribute to the economy. I am not getting the impression that the people who push for AI safety are thinking about this. It seems they are most worried about not losing their position of privilege…

AIUI it with superalignment, it merely means "the AI does what the humans instructing it want it to do". It's a different kind of safety than the built in censoring that most LLMs have.

Re: Safe Superintelligence Inc.

#738

Post by janalsncm »

tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

The scenario where we create an agent that tries and succeeds at outsmarting us in the game of “escape your jail” is the least likely attack vector imo. People like thinking about it in a sort of Silence of the Lambs setup, but reality will probably be far more mundane.

Far more likely is something dumb but dangerous, analogous to the Flash Crash or filter bubbles, emergent properties of relying too much on complex systems, but still powerful enough to break society.

Re: Safe Superintelligence Inc.

#739

Post by fnordpiglet »

fernly wrote: post #733
fnordpiglet wrote: post #68

“””We are assembling a lean, cracked team of the world’s best engineers and researchers dedicated to focusing on SSI and nothing else.””” Cracked indeed

For fun I asked Claude (sonnet) to proofread that paragraph and it found no problem with "cracked team".

Interestingly I asked Opus and it did :-)

Re: Safe Superintelligence Inc.

#740

Post by crowcroft »

roywiggins wrote: post #559
crowcroft wrote: post #383

As others have pointed out, it's the business incentives that create unsafe AI, and this doesn't solve that. Social media recommendation algorithms are already incredibly unsafe for society and young people (girls in particular [1]). When negative externalities exist, government should create regulation that appropriately accounts for that cost. I understand there's a bit of a paradigm shift and new attack vectors wi…

Even without business incentives, the military advantages of AI would inventivize governments to develop it anyway, like they did with nuclear weapons. Nuclear weapons are inherently unsafe, there are some safeguards around them, but they are ultimately dangerous weapons.

I don’t think we should be stopping things from being developed, we just need to acknowledge that externalities exist.
Post reply on HN