Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

371–380 of 1001 posts

Re: Safe Superintelligence Inc.

#371

Post by in3d »

deegles wrote: post #225
TaupeRanger wrote: post #181

Earlier quoted context omitted.

Trying to create "safe superintelligence" before creating anything remotely resembling or approaching "superintelligence" is like trying to create "safe Dyson sphere energy transport" before creating a Dyson Sphere. And the hubris is just a cringe inducing bonus.

'Fearing a rise of killer robots is like worrying about overpopulation on Mars.' - Andrew Ng

Andrew Ng worked on facial recognition for a company with deep ties to the Chinese Communist Party. He’s the absolute worst person to quote.

Re: Safe Superintelligence Inc.

#372

Post by whimsicalism »

kccqzy wrote: post #123
PheonixPharts wrote: post #89

Earlier quoted context omitted.

> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…

Two people I knew recently left Google to join OpenAI. They were solid L5 engineers on the verge of being promoted to L6, and their TC is now $900k. And they are not even doing AI research, just general backend infra. You don't need to be gifted, just good. And of course I can't really fault them for joining a company for the purpose of optimizing TC.

the thing about mentioning compensation numbers on HN is you will get tons of pissy/ressentiment-y replies

Re: Safe Superintelligence Inc.

#373

Post by dindobre »

cosmic_quanta wrote: post #305
tcgv wrote: post #270

Ten years from now will either be: a) Remember all that fuss about AI destroying the world? Lol. ~ or ~ b) I'm so glad those people stepped in to save us from doom! Which one do you think is more likely?

Unless AI starts being 1 000 000x energy efficient, my money is on a). The amount of energy required for AI to be dangerous to its creators is so vast that I can't see how it can realistically happen.

If we manage to harness the ego energy transpiring from some people working on "AI" we should be halfway there!

Re: Safe Superintelligence Inc.

#374

Post by tomrod »

kmacdough wrote: post #369

I'm seeing a lot of criticism suggesting that one company understanding safety won't help what other companies or countries do. This is very wrong. Throughout history, measurement has always been the key to enforcement. The only reason the nuclear test ban treaty didn't ban underground tests was because it couldn't be monitored. In the current landscape there is no formal understanding of what safety means or how it…

> In the current landscape there is no formal understanding of what safety means or how it is achieved. There is no benchmark against which to evaluate ambitious orgs like OpenAI. Anything goes wrong? No one could've known better.

We establish this regularly in the legal sphere, where people seek mediation for harms from systems they don't have liability and control for.

Re: Safe Superintelligence Inc.

#375

Post by orangecat »

cosmic_quanta wrote: post #293
mike_hearn wrote: post #283

Earlier quoted context omitted.

We're really not that far. I'd argue superintelligence has already been achieved, and it's perfectly and knowably safe. Consider, GPT-4o or Claude are: • Way faster thinkers, readers, writers and computer operators than humans are • Way better educated • Way better at drawing/painting ... and yet, appear to be perfectly safe because they lack agency. There's just no evidence at all that they're dangerous. Why isn't t…

Yann LeCun said it best in an interview with Lex Friedman. LLMs don't consume more energy when answering more complex questions. That means there's no inherent understanding of questions. (which you could infer from their structure: LLMs recursively predict the next word, possibly using words they just predicted, and so on).

LLMs don't consume more energy when answering more complex questions.

They can. With speculative decoding (https://medium.com/ai-science/speculative-decoding-make-llm-...) there's a small fast model that makes the initial prediction for the next token, and a larger slower model that evaluates that prediction, accepts it if it agrees, and reruns it if not. So a "simple" prompt for which the small and large models give the same output will run faster and consume less energy than a "complex" prompt for which the models often disagree.

Re: Safe Superintelligence Inc.

#376

Post by almostgotcaught »

doktorhladnjak wrote: post #325
almostgotcaught wrote: post #206

Earlier quoted context omitted.

> their TC is now $900k. Everyone knows that openai TC is heavily weighted by ~~RSUs~~ options that themselves are heavily weighted by hopes and dreams.

When I looked into it and talked to some hiring managers, the big names were offering cash comp similar to total comp for big tech, with stock (sometimes complicated arrangements that were not options or RSUs) on top of that. I’m talking $400k cash for a senior engineer with equity on top.

> big names

Big names where? Inside of openai? What does that even mean?

The only place you can get 400k cash base for senior is quantfi

Re: Safe Superintelligence Inc.

#377

Post by whimsicalism »

015a wrote: post #120
PheonixPharts wrote: post #89

Earlier quoted context omitted.

> compensation packages that truly gifted AI researchers can make now I guess it depends on your definition of "truly gifted" but, working in this space, I've found that there is very little correlation between comp and quality of AI research. There's absolutely some brilliant people working for big names and making serious money, there's also plenty of really talented people working for smaller startups doing incred…

Definitely true of even normal software engineering; my experience has been the opposite of expectations, that TC-creep has infected the industry to an irreparable degree and the most talented people I've ever worked around or with are in boring, medium-sized enterprises in the midwest US or australia, you'll probably never hear of them, and every big tech company would absolutely love to hire them but just can't fig…

perfect sort of thing to say to get lots of upvotes, but absolutely false in my experience at both enterprise and bigtech

Re: Safe Superintelligence Inc.

#378

Post by whimsicalism »

EncomLab wrote: post #217
esafak wrote: post #42

Earlier quoted context omitted.

I think they will easily find enough capable altruistic people for this mission.

I mean SBF was into Altruism - look how that turned out....

and that soured you on altruism as a concept??

i find the way people reason nowadays baffling

Re: Safe Superintelligence Inc.

#380

Post by SSEthrowaway »

klankbrouwerij wrote: post #162

SSI, a very interesting name for a company advancing AI! "Solid State Intelligence" or SSI was also the name of the malevolent entity described in the biography of John C. Lilly [0][1]. It was a network of "computers" (computation-capable solid state systems) that was first engineered by humans and then developed into something autonomous. [0] https://en.wikipedia.org/wiki/John_C._Lilly [1] http://johnclilly.com/

[dead]
Post reply on HN