Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

381–390 of 1001 posts

Re: Safe Superintelligence Inc.

#381

Post by kjkjadksj »

mdp2021 wrote: post #362
kjkjadksj wrote: post #263

Earlier quoted context omitted.

If you want a real technical revolution, you teach the masses how to code their own tailored software, and not just use abstractions and software built by people who sell software to the average user. What a shame we failed at that and are even sliding back in a lot of ways with plummeting technical literacy in smartphone-raised generations.

> you teach the masses how to code their own tailored software That does not seem to be the key recipe to reaching techno-scientific milestones - coders are not necessarily researchers. > plummeting technical literacy in smartphone-raised generations Which shows there are other roots to the problem, given that some of us (many probably in this "club") used our devices generally more productively than said «generation…

Apparently less than half a percent of the worlds population knows how to code. All the software you use, and almost everything you've ever seen with modern technology are generated from this small subpopulation. Now, imagine if that number doubled to 1% of the worlds population. Theoretically there would be as much as twice as much software produced (although less certainly). Now imagine if that number was closer to the world literacy rate of 85%. You think the world wouldn't dramatically change when each and every person can take their given task, job, hobby, whatever, and create helpful software for themselves? I think it would be like The Jetsons.

Re: Safe Superintelligence Inc.

#382

Post by torginus »

PheonixPharts wrote: post #89
eigenvalue wrote: post #15

Glad to see Ilya is back in a position to contribute to advancing AI. I wonder how they are going to manage to pay the kinds of compensation packages that truly gifted AI researchers can make now from other companies that are more commercially oriented. Perhaps they can find people who are ideologically driven and/or are already financially independent. It's also hard to see how they will be able to access enough com…

> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…

Half the advancements around Stable Diffusion (Controlnet etc.) came from internet randoms wanting better anime waifus

Re: Safe Superintelligence Inc.

#383

Post by crowcroft »

As others have pointed out, it's the business incentives that create unsafe AI, and this doesn't solve that. Social media recommendation algorithms are already incredibly unsafe for society and young people (girls in particular [1]).

When negative externalities exist, government should create regulation that appropriately accounts for that cost.

I understand there's a bit of a paradigm shift and new attack vectors with LLMs etc. but the premise is the same imo.

[1] https://nypost.com/2024/06/16/us-news/preteen-instagram-infl...

Re: Safe Superintelligence Inc.

#385

Post by valine »

cosmic_quanta wrote: post #293
mike_hearn wrote: post #283

Earlier quoted context omitted.

We're really not that far. I'd argue superintelligence has already been achieved, and it's perfectly and knowably safe. Consider, GPT-4o or Claude are: • Way faster thinkers, readers, writers and computer operators than humans are • Way better educated • Way better at drawing/painting ... and yet, appear to be perfectly safe because they lack agency. There's just no evidence at all that they're dangerous. Why isn't t…

Yann LeCun said it best in an interview with Lex Friedman. LLMs don't consume more energy when answering more complex questions. That means there's no inherent understanding of questions. (which you could infer from their structure: LLMs recursively predict the next word, possibly using words they just predicted, and so on).

Or alternatively a lot of energy is wasted answering simple questions.

The whole point of the transformer is to take words and iteratively, layer by layer, use the context to refine their meaning. The vector you get out is a better representation of the true meaning of the token. I’d argue that’s loosely akin to ‘understanding’.

The fact that the transformer architecture can memorize text is far more surprising to me than the idea that it might understand tokens.

Re: Safe Superintelligence Inc.

#387

Post by jackothy »

appplication wrote: post #357
jackothy wrote: post #300

Earlier quoted context omitted.

"how someone so smart can be so naive" Do you really think Ilya has not thought deeply about each and every one of your points here? There's plenty of answers to your criticisms if you look around instead of attacking.

I actually do think they have not thought deeply about it or are willfully ignoring the very obvious conclusions to their line of thinking. Ilya has an exceptional ability extrapolate into the future from current technology. Their assessment of the eventual significance of AI is likely very correct. They should then understand that there will not be universal governance of AI. It’s not a nuclear bomb. It doesn’t rely…

It seems like you're saying that if we can't guarantee success then there is no point even trying.

If their assessment of the eventual significance of AI is correct like you say, then what would be your suggested course of action to minimize risk of harm?

Re: Safe Superintelligence Inc.

#388

Post by aristofun »

What a waste of an intelligence.

Pursuing artificial goal to solve a non existent problem to profit off meaningless hype around it.

World would have been better off if he made a decent alternative to k8s or invested his skills into curing cancer or at least protecting world from totalitarian governments and dangerous ideologies (if he wants to belong to vague generic cause).

You know, real problems, like the ones people used to solve back in the old days…

Re: Safe Superintelligence Inc.

#389

Post by hackerlight »

insane_dreamer wrote: post #66

I understand the concern that a "superintelligence" will emerge that will escape its bounds and threaten humanity. That is a risk. My bigger, and more pressing worry, is that a "superintelligence" will emerge that does not escape its bounds, and the question will be which humans control it. Look no further than history to see what happens when humans acquire great power. The "cold war" nuclear arms race, which brough…

China can not win this race and I hate that this comment is going to be controversial among the circle of people that need to understand this the most. It is damn frightening that an authoritarian country is so close to number one in the race to the most powerful technology humanity has invented, and I resent people who push for open source AI for this reason alone. I don't want to live in a world where the first superintelligence is controlled by an entity that is threatened by the very idea of democracy.

Re: Safe Superintelligence Inc.

#390

Post by [deleted] »

vundercind wrote: post #163
arbuge wrote: post #80

Earlier quoted context omitted.

The current intelligences of the world (us) have organized their civilization in a way that the conforming members of society are the norm and criminals the outcasts. Certainly not a perfect system, but something along those lines for the most part. I like to think AGIs will decide to do that too.

Our current meatware AGIs (corporations) are lawless as fuck and have effectively no ethics at all, which doesn’t bode well.

[deleted]
Post reply on HN