Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Safe Superintelligence Inc.

ssi.inc

Post reply on HN
Print view View original

591–600 of 1001 posts

Re: Safe Superintelligence Inc.

#591

Post by oblio »

O_OtheGreat wrote: post #584
internet_co wrote: post #538

Earlier quoted context omitted.

"In chess" for AI papers == "in mice" for medical papers. Against lichess levels 1, 2, 5, which use a severely dumbed down Stockfish version. Of course it is possible that SSI has novel, unpublished ideas.

Lichess 5 is better than the vast majority of chess players

I think the main point is that from a human intelligence perspective chess is easy mode. Clearly defined, etc.

Think of politics or general social interactions for actual hard mode problems.

Re: Safe Superintelligence Inc.

#592

Post by Kerb_ »

majormajor wrote: post #519
foolishbard wrote: post #500

Earlier quoted context omitted.

There's a chance that these systems can actually out perform their training data and be better than the sum of their parts. New work out Harvard talks about this idea of "transcendence" https://arxiv.org/abs/2406.11741 While this is a new area, it would be naive to write this off as just science fiction.

It would be nice if authors wouldn't use a loaded-as-fuck word like "transcendence" for "the trained model can sometimes achieve better performance than all [chess] players in the dataset" because while certainly that's demonstrating an impressive internalization of the game, it's also something that many humans can also do. The machine, of course, can be scaled in breadth and performance, but... "transcendence"? Are…

It transcends the training data, I get the usage intended but it certainly is ripe for misinterpretation

Re: Safe Superintelligence Inc.

#593

Post by vineyardmike »

reissbaker wrote: post #577
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

I'm pretty sure "Altman and company" don't have much to do with this — this is Ilya, who pretty famously tried to get Altman fired, and then himself left OpenAI in the aftermath. Ilya is a brilliant researcher who's contributed to many foundational parts of deep learning (including the original AlexNet); I would say I'm somewhat pessimistic based on the "safety" focus — I don't think LLMs are particularly dangerous,…

> I don't think LLMs are particularly dangerous

“Everyone” who works in deep AI tech seems to constantly talk about the dangers. Either they’re aggrandizing themselves and their work, or they’re playing into sci-fi fear for attention or there is something the rest of us aren’t seeing.

I’m personally very skeptical there is any real dangers today. If I’m wrong, I’d love to see evidence. Are foundation models before fine tuning outputting horrific messages about destroying humanity?

To me, the biggest dangers come from a human listening to a hallucination and doing something dangerous, like unsafe food preparation or avoiding medical treatments. This seems distinct from a malicious LLM super intelligence.

Re: Safe Superintelligence Inc.

#594

Post by mewpmewp2 »

mjr00 wrote: post #568
ohcmon wrote: post #494

Earlier quoted context omitted.

> glorified token predicting machine trained on existing data (made by humans) sorry to disappoint, but human brain fits the same definition

If you genuinely believe your brain is just a token prediction machine, why do you continue to exist? You're just consuming limited food, water, fuel, etc for the sake of predicting tokens, like some kind of biological crypto miner.

Well, yes. I won't commit suicide though, since it is an evolutionarily developed trait to keep living and reproducing since only the ones with that trait survive in the first place.

Re: Safe Superintelligence Inc.

#595

Post by TaylorAlexander »

flockonus wrote: post #588
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

Likewise, i'm baffled by intelligent people [in such denial] still making the reductionist argument about token prediction being a banal ability. It's not. It's not very different than how our intelligence manifest.

> It's not very different than how our intelligence manifest.

[citation needed]

Re: Safe Superintelligence Inc.

#596

Post by JumpCrisscross »

mjr00 wrote: post #550
alecco wrote: post #533

Earlier quoted context omitted.

Because it's likely soon LLMs will be able to teach themselves and surpass humans. No consciousness, no will. But somebody will have their power. Dark government agencies and questionable billionaires. Who knows what will it enable them to do. https://en.wikipedia.org/wiki/AlphaGo_Zero

Mind defining "likely" and "soon" here? Like 10% chance in 100 years, or 90% chance in 1 year? Not sure how a Go engine really applies. Do you consider cars superintelligent because they can move faster than any human?

> Mind defining "likely" and "soon" here? Like 10% chance in 100 years, or 90% chance in 1 year?

We're just past the Chicago pile days of LLMs [1]. Sutsever believes Altman is running a private Manhattan project in OpenAI. I'd say the evidence for LLMs having superintelligence capability is on shakier theoretical ground today than nuclear weapons were in 1942, but I'm no expert.

Sutsever is an expert. He's also conflicted, both in his opposition to OpenAI (reputationally) and pitching of SSI (financially).

So I'd say there appears to be a disputed but material possibility of LLMs achieving something that, if it doesn't pose a threat to our civilisation per se, does as a novel military element. Given that risk, it makes sense to be cautious. Paradoxically, however, that risk profile calls for strict regulation approaching nationalisation. (Microsoft's not-a-taker takeover of OpenAI perhaps providing an enterprising lawmaker the path through which to do this.)

[1] https://en.wikipedia.org/wiki/Chicago_Pile-1

Re: Safe Superintelligence Inc.

#597

Post by marshray »

alecco wrote: post #540
tiarafawn wrote: post #403

If superintelligence can be achieved, I'm pessimistic about the safe part. - Sandboxing an intelligence greater than your own seems like an impossible task as the superintelligence could potentially come up with completely novel attack vectors the designers never thought of. Even if the SSI's only interface to the outside world is an air gapped text-based terminal in an underground bunker, it might use advanced psych…

We are far from a conscious entity with willpower and self preservation. This is just like a calculator. But a calculator that can do things that will be like miracles to us humans. I worry about dangerous humans with the power of gods, not about artificial gods. Yet.

> Conscious entity... willpower

I don't know what that means. Why should they matter?

> Self preservation

This is no more than a fine-tuning for the task, even with current models.

> I worry about dangerous humans with the power of gods, not...

There's no property of the universe that you only have one thing to worry about at a time. So worrying about risk 'A' does not in any way allow us to dismiss risks 'B' through 'Z'.

Re: Safe Superintelligence Inc.

#598

Post by mjr00 »

mewpmewp2 wrote: post #594
mjr00 wrote: post #568

Earlier quoted context omitted.

If you genuinely believe your brain is just a token prediction machine, why do you continue to exist? You're just consuming limited food, water, fuel, etc for the sake of predicting tokens, like some kind of biological crypto miner.

Well, yes. I won't commit suicide though, since it is an evolutionarily developed trait to keep living and reproducing since only the ones with that trait survive in the first place.

If LLMs and humans are the same, should it be legal for me to terminate you, or illegal for me to terminate an LLM process?

Re: Safe Superintelligence Inc.

#599

Post by EGreg »

I'm going to come out and state the root of the problem.

I can't trust remote AI, any more than I can trust a remote website.

If someone else is running the code, they can switch it up anytime. Imagine trusting someone who simulates everything you need to trust them, giving them all your private info, and then screws you over in an instant. AI is capable of it far more than biological beings with "costly signals".

If it's open source, and I can run it locally, I can verify that it doesn't phone home, and the weights can be audited by others.

Just like I wouldn't want to spend 8 hours a day in a metaverse owned by Zuck, or an "everything app" owned by Elon, why would I want to give everything over to a third party AI?

I like Ilya. I like Elon. I like Moxie Marlinspike and Pavel Durov. But would I trust their companies to read all my emails, train their data on them, etc.? And hope nothing leaks?

And of course then there is the issue of the AI being trained to do what they want, just like their sites do what they want, which in the case of Twitter / Facebook is not healthy for society at large, but creates angry echo chambers, people addicted to stupid arguing and videos.

I think there have to be standards for open source AI, and something far better than Langchain (which sucks). This is what I think it should look like: https://engageusers.ai/ecosystem.pdf -- what do you think?

Re: Safe Superintelligence Inc.

#600

Post by Yoric »

reissbaker wrote: post #577
chrisldgk wrote: post #473

Not to be too pessimistic here, but why are we talking about things like this? I get that it’s a fun thing to think about, what we will do when a great artificial superintelligence is achieved and how we deal with it, feels like we’re living in a science fiction book. But, all we’ve achieved at this point is making a glorified token predicting machine trained on existing data (made by humans), not really being able t…

I'm pretty sure "Altman and company" don't have much to do with this — this is Ilya, who pretty famously tried to get Altman fired, and then himself left OpenAI in the aftermath. Ilya is a brilliant researcher who's contributed to many foundational parts of deep learning (including the original AlexNet); I would say I'm somewhat pessimistic based on the "safety" focus — I don't think LLMs are particularly dangerous,…

I actually feel that they can be very dangerous. Not because of the fabled AGI, but because

1. they're so good at showing the appearance of being right;

2. their results are actually quite unpredictable, not always in a funny way;

3. C-level executives actually believe that they work.

Combine this with web APIs or effectors and this is a recipe for disaster.

Post reply on HN