Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are you so sure I'll lose?

twitter.com

Post reply on HN
Print view View original

81–90 of 90 posts

Re: Why are you so sure I'll lose?

#81

Post by tpm »

mitthrowaway2 wrote: post #71
tpm wrote: post #55

Earlier quoted context omitted.

>> AI doesn't act by itself > It can be made to quite easily. Not really, no. You can of course create an acting-by-itself bot but it will run into limits very very soon, because the world model and planning parts of AI are in their infant stages, if that.

The concern is not about GPT-4. It's about what 10 years of research gets you to when, five years ago, it seemed completely apparent that even GPT-4 was 50 years off.

GPT-1 was 5 years ago, GPT-2 was 4 years ago and by that time it was quite clear to some that this will scale soon.

But I'm not talking about the GPT-4 either. Many decades and billions poured into research still didn't give us autonomous driving and there is no clear path how that should happen. So I'm not really holding my breath for anything even more capable.

Re: Why are you so sure I'll lose?

#82

Post by yawpitch »

dbuxton wrote: post #56
yawpitch wrote: post #52

Earlier quoted context omitted.

Sure… imagine, for example, health insurance companies employing proprietary ML models to decide if you’re covered, or for-profit prisons using ML models to decide if someone will be a recidivist. Or governments or law enforcement or private corporations or simply rogue free speech absolutist billionaires using image recognition and ML models to decide which pixels represent a combat age male? The AI we’ve got right…

Ah ok I see what you mean. I think what we have today (weak ML) are models which can be trained to operate at the quality of a human for specific tasks. So I am definitely terrified by the recidivism example, because it’s not obvious to me that there is a human who can do that task, but I’m pretty relaxed about the coverage one (because presumably at least in principle if you automate these sorts of manual tasks you…

In all three cases the ML model can only every perform as well as a human with no internal morality whatsoever, no ability to consider the ethics of their actions or their consequences, and fully (and completely) inculcated with the explicit and implicit biases and incentives of their employer.

Trusting an ML model in any of these cases is guaranteeing the worst case possible scenario… at least the human in the loop MIGHT choose not to act in its employer’s interests.

Re: Why are you so sure I'll lose?

#83

Post by yawpitch »

philipswood wrote: post #60
yawpitch wrote: post #51

Earlier quoted context omitted.

For one, the child can play any number of other games or choose not to play at all; the child can break the board, flip it upside down, fold it in half, introduce the concept of wraparound or flip the way pieces move… the child is free to choose to change the definition and rules of the game… the AI can only, ever, play chess, using only the rules it was given. The metaphor assumes that life is a game in which there…

I think his message is targeting a bit deeper than that. If I had so sum it up (always hard with an allegory): Developing superhuman AI is a game you can only play once and can expect to lose without ever having a good understanding of how, why or even when you lost. The suggestion is (like in the movie Wargames) that the best move is not to play. I'd say the child in the story is mankind. Developing strong AI is the…

Interesting interpretation, but if true it’s poorly constructed: the child can’t be mankind, as the “you” in the game would then be god and god (not mankind) would have developed the (in the story already extant) AI… and AI that is not, obviously, a strong AGI.

Now, it could be that “you” is Homo Not So Sapiens and the child is Homo Antesingularity… but even then I think you’re stretching it past its breaking points.

Re: Why are you so sure I'll lose?

#84

Post by nvm0n2 »

aerojoe23 wrote: post #58
jacquesm wrote: post #47

Earlier quoted context omitted.

Rock climbing = personal entertainment. That's why rock climbers don't need to worry about Stockfish. But competitive chess players do have to worry about Stockfish.

There is competitive rock climbing. Rock climbing isn't all about physical strength, you have to pick your route. Eventually some military robot company will want to prove that its combat bot can out perform humans on any terrain. And we'll see best humans vs matches that can match and a few years later beat their performance. Of course the professional competitors don't have to worry about competing against a human…

How is that a unique threat vs steroids?

Re: Why are you so sure I'll lose?

#85

Post by nvm0n2 »

philipswood wrote: post #60
yawpitch wrote: post #51

Earlier quoted context omitted.

For one, the child can play any number of other games or choose not to play at all; the child can break the board, flip it upside down, fold it in half, introduce the concept of wraparound or flip the way pieces move… the child is free to choose to change the definition and rules of the game… the AI can only, ever, play chess, using only the rules it was given. The metaphor assumes that life is a game in which there…

I think his message is targeting a bit deeper than that. If I had so sum it up (always hard with an allegory): Developing superhuman AI is a game you can only play once and can expect to lose without ever having a good understanding of how, why or even when you lost. The suggestion is (like in the movie Wargames) that the best move is not to play. I'd say the child in the story is mankind. Developing strong AI is the…

> Developing superhuman AI is a game you can only play once

But that's not right. Stockfish is superhumanly intelligent at playing chess but it is not the result of a single match between its creator and itself. Humans played chess against computers over and over, for decades, and the "superintelligence" level was seen coming a mile off. Moreover it was never dangerous and those AI techniques never led to a general intelligence.

This is the problem with AI doomers like Yudkowsky. They use rhetorical skill to make clever sounding arguments that contain fallacies, and hope you won't notice. The assumption you can only "play once" is a wild one and thrown in at the end with absolutely nothing earlier in the story supporting it or backing it up.

Re: Why are you so sure I'll lose?

#86

Post by nvm0n2 »

navane wrote: post #37

Why does the young kid only get to play once? It's not as it'll beat StockFish after any amount of games?

The guy who wrote the post is an AI doomer who believes that the moment AI becomes "general" it will recursively improve itself into being a god and then immediately kill everyone, and that will happen so fast that there won't be any time to switch it off. Hence only one play.

The idea is nonsensical and the ending of the story is also nonsensical as a consequence, but that's what it's about.

Re: Why are you so sure I'll lose?

#87

Post by jacquesm »

NoGravitas wrote: post #78
jacquesm wrote: post #72

Earlier quoted context omitted.

Depending on where you live climate change may very well be a today thing. Kessler syndrome not so much. Guys with guns and missiles: very much a today thing.

I mean, via Starlink, Musk is working pretty hard on making Kessler Syndrome a today thing as fast as possible.

Not in LEO.

Re: Why are you so sure I'll lose?

#88

Post by yowlingcat »

xcode42 wrote: post #75
lucubratory wrote: post #62

Earlier quoted context omitted.

What specific demonstrations of capability by AI would convince you that your proposed timeline for reaching AGI isn't correct, and that it will happen significantly sooner? If it helps to think about it, there's a good DeepMind paper on a relatively concrete scaling system on the path to AGI: https://arxiv.org/abs/2311.02462

Thank you for the paper, this seems awesome, I'll have a look later. Deepmind pretty consistently comes up with interesting ai research results, so this should be pretty neat at the very least. Regarding what would convince me that AGI is around the corner, I guess I should start by defining what I'm thinking of when I say AGI, because to be fair AGI is pretty vague. I expect it to be an agent capable of independent…

I find this model pretty fascinating. Said another way, there are two aspects necessary for AGI as you see it:

- Independence + continual improvement (it's always either doing something useful whether you tell it to or not)

- Reliable, prompt and accurate learning of new material

The first one is interesting because it's theoretically possible to try right now (just put an LLM scanning in a background job), but clearly not trivial.

But I think the second issue you raise is the more fundamental one. There have been a couple papers I've seen that have demonstrated that most major LLMs struggle with recalling things in a way that makes use of equivalence and substitution -- for example, if I tell GPT that X is Y's son, then if it will know that if I ask who is the father of Y, it understands the answer is X. If it cannot show basic competence in equivalence and substitution, then it begins to break through the 4th wall of a "sentient entity" and phenomenologically reveal its true colors as a stochastic parrot.

It's fascinating. There are conceptual issues, as you mention, and they're not just theoretical, but they're very noticeable (for now).

Here's the question I would ask you -- do you think even if we're no where near AGI, the world has fundamentally changed since the mass distribution of LLMs? I have felt a noticeable shift as the technology's adoption is beginning to alter how society conceives of and digests media. It feels like a discrete new evolutionary state of the internet. Social media fabric feels like it's in a new phase compared to even a year ago as new weapons of mass production have been released.

Re: Why are you so sure I'll lose?

#89

Post by loup-vaillant »

chrismatheson wrote: post #23
loup-vaillant wrote: post #15

Earlier quoted context omitted.

It’s not about motivation, it’s about doing whatever it’s been programmed to do, and grabbing whatever means it can to that end. Now something as puny as a large language model won’t have much reach (the current ones can’t even read the internet after the initial learning phase), but if you have a way to give the thing an unbounded goal, then it will use all means at its disposal to make that goal happen. Heck, even…

Are you suggesting the earth may be one giant computer tasked with finding the ultimate question ?

I’m suggesting the earth may be turned into one giant computer tasked with finding a well defined question.

The one posed in HG2G’s would likely just return a syntax or type error.

Re: Why are you so sure I'll lose?

#90

Post by xcode42 »

yowlingcat wrote: post #88
xcode42 wrote: post #75

Earlier quoted context omitted.

Thank you for the paper, this seems awesome, I'll have a look later. Deepmind pretty consistently comes up with interesting ai research results, so this should be pretty neat at the very least. Regarding what would convince me that AGI is around the corner, I guess I should start by defining what I'm thinking of when I say AGI, because to be fair AGI is pretty vague. I expect it to be an agent capable of independent…

I find this model pretty fascinating. Said another way, there are two aspects necessary for AGI as you see it: - Independence + continual improvement (it's always either doing something useful whether you tell it to or not) - Reliable, prompt and accurate learning of new material The first one is interesting because it's theoretically possible to try right now (just put an LLM scanning in a background job), but clear…

Regarding the first one, it's actually a surprisingly difficult problem. Sure we can try right now but deep learning systems currently have a lot of issues with continuous learning, if you just do it the naive way they keep forgetting all they learned to get better performance on the most recent examples. But yeah the second one definitely seems like the harder of the two. The building blocks of intelligence and the mind are indeed endless fascinating, I spend way too much time thinking about this stuff.

Oh no I agree, the current AI systems have already caused notable changes in society and even if the development of even more powerful AI stagnates here(for now) we are bound to continue to see societal changes for some time as this is all quite new and society is still adapting and figuring out the "right way" to handle all this. In particular the impact it has had on artists of all kinds is widespread and I'll be curious to see how it pans out and how we handle it as a society.

Post reply on HN