Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

921–930 of 1001 posts

Re: Safe Superintelligence Inc.

#921

Post by __MatrixMan__ »

creato wrote: post #459
__MatrixMan__ wrote: post #445

Earlier quoted context omitted.

Government is controlled by the highest bidder. I think we should be prepared to do this ourselves by refusing to accept money made by unsafe businesses, even if it means saying goodbye to the convenience of fungible money.

"Government doesn't work. We just need to make a new government that is much more effective and far reaching in controlling people's behavior."

Banding together and refusing to accept harmful money is indeed akin to creating a government, and would indeed be more effective at controlling people's behavior.

But participation would be voluntary, and the restriction of harmful behavior would apply to it's enemies, not its citizens. So I'm not quite sure what the problem is.

Re: Safe Superintelligence Inc.

#922

Post by troupo »

343242dfsdf wrote: post #888
ilrwbwrkhv wrote: post #224

Earlier quoted context omitted.

There is no "superintelligence" or "AGI". People are falling for marketing gimmicks. These models will remain in the word vector similarity phase forever. Till the time we understand consciousness, we will not crack AGI and then it won't take brute forcing of large swaths of data, but tiny amounts. So there is nothing to worry. These "apps" might be as popular as Excel, but will go no further.

> There is no "superintelligence" or "AGI" There is intelligence. The LLM current state-of-the-art technology produces output analog to natural intelligences. This things are already intelligent. Saying that LLMs aren't producing "intelligence" is like saying planes actually don't fly because they are not flapping their wings like birds. If you run fast enough, you'll end flying at some point. Maybe "intelligence" is…

> There is intelligence.

There isn't

> Maybe "intelligence" is just enough statistics and pattern prediction, till the point you just say "this thing is intelligent".

Even the most stupid people can usually ask questions and correct their answers. LLMs are incapable of that. They can regurgitate data and spew a lot of generated bullshit, some of which is correct. Doesn't make them intelligent.

Here's a prime example that appeared in my feed today: https://x.com/darthsidius1985/status/1802423010886058254 And all the things wrong with it: https://x.com/yvanspijk/status/1802468042858737972 and https://x.com/yvanspijk/status/1802468708193124571

Intelligent it is not

Re: Safe Superintelligence Inc.

#923

Post by Freebytes »

JW_00000 wrote: post #871
rstuart4133 wrote: post #841

Earlier quoted context omitted.

AGI does look like an unsolved problem right now, and a hard one at that. But I think it is wrong to think that it needs an AGI to cause total havoc. I think my dyslexic namesake Prof Stuart Russell got it right. It humans won't need an AGI to dominate and kill each other. Mosquitoes have killed far more people than war. Ask yourself how long will it take us to develop a neutral network as smart as a mosquito, becaus…

But then, haven't we reached that point already with the development of nuclear weapons? I'm more scared of a lunatic (whether of North Korean, Russian, American, or any other nationality) being behind the "nuclear button" than an artificial mosquito swarm.

You cannot copy a nuclear weapon via drag and drop.

Re: Safe Superintelligence Inc.

#924

Post by dreamcompiler »

rstuart4133 wrote: post #841
dreamcompiler wrote: post #756

Earlier quoted context omitted.

AGI is still a long way off. The history of AI goes back 65 years and there have been probably a dozen episodes where people said "AGI is right around the corner" because some program did something surprising and impressive. It always turns out human intelligence is much, much harder than we think it is. I saw a tweet the other day that sums up the current situation perfectly: "I don't need AI to paint pictures and w…

AGI does look like an unsolved problem right now, and a hard one at that. But I think it is wrong to think that it needs an AGI to cause total havoc. I think my dyslexic namesake Prof Stuart Russell got it right. It humans won't need an AGI to dominate and kill each other. Mosquitoes have killed far more people than war. Ask yourself how long will it take us to develop a neutral network as smart as a mosquito, becaus…

Producing a mosquito brain is easy. Powering it with the Krebs cycle is much harder.

Yes you can power these things with batteries. But those are going to be a lot bigger than real mosquitos and have much shorter flight times.

Re: Safe Superintelligence Inc.

#925

Post by troupo »

343242dfsdf wrote: post #910
antihipocrat wrote: post #565

Earlier quoted context omitted.

It's a massive attractor of investment funding. Is it proven to be massively profitable?

Being profitable is probably a matter of time and technology maturing. Think about the first Iphone, Windows 95, LCD/LEDs, etc. The potential of a sufficiently intelligent agent, probably something very close to a really good AGI, albeit still not an ASI, could be measured in billions of billions of mostly inmediate return of investment. LLMs are already well into the definition of hard AI, there are already strong s…

> Being profitable is probably a matter of time and technology maturing. Think about the first Iphone, Windows 95, LCD/LEDs, etc.

Juicero, tulips....

> then you "just" assemble an AI team, and start training bigger, better LLMs, ASAP, AFAP.

There's a limit to LLMs and we may have reached it. Both physical: there is not enough capacity in the world to train bigger models. And data-related: once you've gobbled up most of internet, movies and visual arts, there's an upper limit on how much better these models can become.

Re: Safe Superintelligence Inc.

#926

Post by molszanski »

If anyone thinks we can build SAFE superintelligence, I think they are wrong.

I highly, highly recommend this book "Superintelligence" by Nick Bostrom

https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...

It has one of the highest information and idea density per paragraph I've read.

And it is full good ideas. Honestly, reading any discussion or reasoning about AGI / Safe AGI is kinda pointless after that.

Author has described all possible decisions and paths and we will follow just one of them. Could find a single flaw in his reasoning in any history branch.

Re: Safe Superintelligence Inc.

#927

Post by snaeker58 »

If I’d get a penny every time I heard someone say “AGI is around the corner” I’d have enough money to build myself a house on mars or to afford a 2 room apartment in NYC. Add the Pennies from “Sustainable fusion around the corner” people and I could afford both.

Re: Safe Superintelligence Inc.

#928

Post by visarga »

slashdave wrote: post #129
visarga wrote: post #20

Earlier quoted context omitted.

I have a strong intuition that chat logs are actually the most useful kind of data. They contain many LLM outputs followed by implicit or explicit feedback, from humans, from the real world, and from code execution. Scaling this feedback to 180M users and 1 trillion interactive tokens per month like OpenAI is a big deal.

Except LLMs are a distraction from AGI

If brain without language would suffice, a single human could rediscover all we know on their own. But it's not like that, brains are feeble individually, only in societies we have cultural evolution. If humanity lost language and culture and start from scratch, it would take us another 300K years to rediscover what we lost.

But if you train a random-init LLM on the same data, it responds (almost) like a human on a diversity of tasks. Does that imply humans are just language models on two feet? Maybe we are also language modelling our way through life. New situation comes up, we generate ideas based on language, select based on personal experience, and then act and observe the outcomes to update our preferences in the future.

Re: Safe Superintelligence Inc.

#929

Post by ghodith »

jrflowers wrote: post #764
jpk wrote: post #705

Earlier quoted context omitted.

Since nobody bothered to explain their downvotes, I'll say it: This is a pretty uncharitable reading of the quoted comment.

This is a good point. When you read this: > Instagram’s algorithms have steered men with deviant sexual attraction to children to the girl’s page, flooding it with unwanted comments and come-ons, according the Journal. It is a normal response to think “I wonder what was so attractive about the images that the child made that these men were forced to follow and comment on them. I will post about how we shouldn’t judge…

No one mentioned wanting to see them until you..

Re: Safe Superintelligence Inc.

#930

Post by acjohnson55 »

sneak wrote: post #842
roywiggins wrote: post #559

Earlier quoted context omitted.

Even without business incentives, the military advantages of AI would inventivize governments to develop it anyway, like they did with nuclear weapons. Nuclear weapons are inherently unsafe, there are some safeguards around them, but they are ultimately dangerous weapons.

Dangerous weapons are not inherently unsafe. Take a Glock, for example. It is a deadly weapon, designed for one thing and one thing alone. It is, however, one of the safest machines ever built.

"Safe", in what sense?
Post reply on HN