Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You are the scariest monster in the woods

jamie.ideasasylum.com

Post reply on HN
Print view View original

341–350 of 405 posts

Re: You are the scariest monster in the woods

#341

Post by nonethewiser »

omnicognate wrote: post #295
nonethewiser wrote: post #270

Earlier quoted context omitted.

Is it too brief to capture it? Here is a one sentence statement I found from one of his slides: >Turing’s version of Gödel’s theorem tells us that, for any set of mechanical theorem-proving rules R, we can construct a mathematical statement G(R) which, if we believe in the validity of R, we must accept as true; yet G(R) cannot be proved using R alone. I have no doubt the books are good but the original comment asked…

That's a summary of Godel's theorem, which nobody disputes, not of Penrose's argument that it implies computers cannot emulate human intelligence. I'm really not trying to evade further discussion. I just don't think I can sum that argument up. It starts with basically "we can perceive the truth not only of any particular Godel statement, but of all Godel statements, in the abstract, so we can't be algorithms because…

Thank you for taking the time to clarify. Lots to chew on here.

Re: You are the scariest monster in the woods

#342

Post by ACCount37 »

bangaroo wrote: post #205
myrmidon wrote: post #30

Earlier quoted context omitted.

If you make the same argument for flight it looks really weak. Cognition is (to me) not even the most impressive and out-of-reach achievement: That would be how (our) and animals bodies are self-assembling, self-repairing and self-replicating, with an impressive array of sensors and actors in a highly integrated package. I honestly believe our current technology is much closer to emulating a human brain than it is to…

> if you make the same argument for flight it looks really weak. flight is an extremely straightforward concept based in relatively simple physics where the majority of the critical, foundational ideas involved were already near-completely understood in the late 1700s. i really don't think it's fair to compare the two

I'm sure that intelligence is an extremely straightforward concept based in relatively simple math where the majority of the critical, foundational ideas involved were already near-completely understood in the late 1900s.

If you read about in a textbook from year 2832, that is.

Re: You are the scariest monster in the woods

#343

Post by johnnyanmac »

myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

I don't dismiss the idea of faster than light travel, and AFAIK we have no way to confirm that outside of ideas (and simply ideas) like wormholes or other cheats to "fold space".

I don't dismiss AI. But I do dismiss what is currently sold to me. It's the equivalent of saying "we made a rocket that can go mach 1000!". That's impressive. But we're still 2-3 magnitudes off from light speed. So I will still complain about the branding despite some dismissals of "yea but imagine in another 100 years!". It's not about semantics so much as principle.

That's on top of the fact that we'd only starting to really deal with significant time dilation by that point, and we know it'll get more severe as we iterate. What we're also not doing is using this feat to discuss how to address those issues. And that's the real frustrating part.

Re: You are the scariest monster in the woods

#344

Post by johnnyanmac »

j2kun wrote: post #221
woeirua wrote: post #44

There's this weird disconnect in tech circles, where everyone is deathly afraid of AGI, but totally asleep on the very real possibility of thermonuclear war breaking out in Europe or Asia over the next 10 years. There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control. AGI might happen, but the threat of nuclear war…

> everyone is deathly afraid of AGI I think this is a vast overstatement. A small group of influential people are deathly afraid of AGI, or at least using that as a pretext to raise funding. But I agree that there are so many more things we should be deathly afraid of. Climate change tops my personal list as the biggest existential threat to humanity.

I sure wish my conspiracy theories could lead to me running billion dollar projects to defend against my shadow demons. Instead I just get ratio'd by the internet and get awkward silences at family gatherings.

I think the sad part is that most people in power aren't planning to be around in 10 years, so they don't care about any long term issues that are cropping up. leave it to their grandchildren to burn with the world.

Re: You are the scariest monster in the woods

#345

Post by johnnyanmac »

nonethewiser wrote: post #294
aradox66 wrote: post #284

Earlier quoted context omitted.

It's not an assumption, it's an extremely developed international field of tactical and strategic study that leads to these conclusions

Extremely developed thought experiment maybe. Only 2 nuclear weapons have ever been dropped. Which is why I say it's a massive assumption. You tell me. How does this escalate into a total destruction scenario? Russia uses a small nuke on a military target in the middle of nowhere Ukraine. ___________________________. Everyone is firing nukes at eachother. Fill in the blank. We are not talking about the scenario where…

>How does this escalate into a total destruction scenario?

My favorite historical documentary: https://www.youtube.com/watch?v=Pk-kbjw0Y8U (my new favorite part is America realizing "fuck, we're dumasses" far too late into the warfare they started).

That is to say: you're assuming a lot of good faith in a time of unrest with several leaders looking for any excuse enact martial law. For all we know, the blank is "Trump overreacts and authorizes a nuclear strike on Los Angeles"(note the word "authorizes". Despite the media, the president cannot unilaterally fire a nuclear warhead). That bizarre threat alone might escalate completely unrelated events and boom. Chaos.

Re: You are the scariest monster in the woods

#346

Post by johnnyanmac »

myrmidon wrote: post #55
woeirua wrote: post #44

There's this weird disconnect in tech circles, where everyone is deathly afraid of AGI, but totally asleep on the very real possibility of thermonuclear war breaking out in Europe or Asia over the next 10 years. There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control. AGI might happen, but the threat of nuclear war…

I personally think that AI is a realistic extinction threat for our whole species within this century, and a full nuclear war never was (and probably never will be). Neither is climate change. Collapse of our current civilizations? Sure. Extinction? No. And I honestly see stronger incentives on a road towards us being outcompeted by AI, then on our leaders starting a nuclear war.

Total extinction of any dominant species is really hard. Very few post-apocalyptic settings suggest a full extinction and usually show some thousands of survivors struggling with the new norm. Humans in particular are very adaptable so thorough killing all 8 billion of us would be difficult no matter the scenario. I think only the Sun can do that and that's assuming we fail to find an exit strategy 5 billion years in (we're less than a thousandth of a percent into humanity if we measure on that scale).

As such, I'd say "extinction" is more of a colloquial use of "Massive point in history that kills off billions in short order".

Re: You are the scariest monster in the woods

#347

Post by nonethewiser »

johnnyanmac wrote: post #345
nonethewiser wrote: post #294

Earlier quoted context omitted.

Extremely developed thought experiment maybe. Only 2 nuclear weapons have ever been dropped. Which is why I say it's a massive assumption. You tell me. How does this escalate into a total destruction scenario? Russia uses a small nuke on a military target in the middle of nowhere Ukraine. ___________________________. Everyone is firing nukes at eachother. Fill in the blank. We are not talking about the scenario where…

>How does this escalate into a total destruction scenario? My favorite historical documentary: https://www.youtube.com/watch?v=Pk-kbjw0Y8U (my new favorite part is America realizing "fuck, we're dumasses" far too late into the warfare they started). That is to say: you're assuming a lot of good faith in a time of unrest with several leaders looking for any excuse enact martial law. For all we know, the blank is "Trum…

I think this perfectly demonstrates my point that the path from isolated tactical nuke to wide scale nuclear war is quite unclear and by no means necessary. Thank you.

Re: You are the scariest monster in the woods

#348

Post by johnnyanmac »

nonethewiser wrote: post #347
johnnyanmac wrote: post #345

Earlier quoted context omitted.

>How does this escalate into a total destruction scenario? My favorite historical documentary: https://www.youtube.com/watch?v=Pk-kbjw0Y8U (my new favorite part is America realizing "fuck, we're dumasses" far too late into the warfare they started). That is to say: you're assuming a lot of good faith in a time of unrest with several leaders looking for any excuse enact martial law. For all we know, the blank is "Trum…

I think this perfectly demonstrates my point that the path from isolated tactical nuke to wide scale nuclear war is quite unclear and by no means necessary. Thank you.

I wish it was a clear path. That's the scariest part. Remember that one assassation escalated to The Great War.

It'll be a similar flimsy straw breaking that will mark the start of nuclear conflict after years of rising tensions. And by then pandora's box will be opened.

Re: You are the scariest monster in the woods

#349

Post by myrmidon »

lo_zamoyski wrote: post #314
myrmidon wrote: post #276

Earlier quoted context omitted.

I'm just assuming materialism, and that assumption is basically for complete lack of convincing alternatives (to me). With "agency" I just mean the ability to affect the physical world (not some abstract internal property). Regarding "computers have no concepts of things": I'm happy with the "meaning" of something being a fuzzy cloud in some high dimensional space, and consider this plausible/workable both for our mi…

This is all very hand wavy. You don't address in the least what I've written. My criticisms stand. Otherwise... > I'm just assuming materialism, and that assumption is basically for complete lack of convincing alternatives (to me). What do you mean by "materialism"? Materialism has a precise meaning in metaphysics (briefly, it is the res extensa part of Cartesian dualism with the res cogitans lopped off). This brand…

By "materialism" I mean that human cognition is simply an emergent property of purely physical processes in (mostly) our brains.

All the individual assumptions basically come down to that same point in my view.

1) Human intelligence is entirely explicable in evolutionary terms

What would even be the alternative here? Evolution plots out a clear progression from something multi-cellular (obviously non-intelligent) to us.

So either you need some magical mechanism that inserted "intelligence" at some point in our species recent evolutionary past, or an even wilder conspiracy theory (e.g. "some creator built us + current fauna exactly, and just made it look like evolution").

2) Intelligence strictly biological

Again, this is simply not an option if you stick to materialism in my view. you would need to assume some kind of bio-exclusive magic for this to work.

3) Silicon is somehow intrinsically bound up with computation

I don't understand what you mean by this.

> It can't even explain color

Perceiving color is just how someones brain reacts to a stimulus? Why are you unhappy with that? What would you need from a satisfactory explanation?

I simply see no indicator against this flavor of materialism, and everything we've learned about our brains so far points in favor.

Thinking, for us, results in and requires brain activity, and physically messing with our brains operation very clearly influences the whole spectrum of our cognitive capabilities, from the ability to perceive pain, color, motion, speech to consciousness itself.

If there was a link to something metaphysical in every persons brain, then I would expect at least some favorable indication before entertaining that notion, and I see none (or some plausible mechanism at the very least).

Re: You are the scariest monster in the woods

#350

Post by slow_typist »

myrmidon wrote: post #293
slow_typist wrote: post #149

Earlier quoted context omitted.

In short, by definition, computers are symbol manipulating devices. However complex the rules of symbol manipulation, it is still a symbol manipulating device, and therefore neither intelligent nor sentient. So AGI on computers is not possible.

This is not an argument at all, you just restate your whole conclusion as an assumption ("a symbol manipulating device is incapable of cognition"). It's not even a reasonable assumption (to me), because I'd assume an exact simulation of a human brain to have the exact same cognitive capabilities (which is inevitable, really, unless you believe in magic). And machines are well capable of simulating physics. I'm not ad…

You can assume whatever you want to, but if you were right, than the human brain itself would be nothing more than a symbol manipulating device. While that is not necessarily a falsifiable stance, the really interesting questions are what is consciousness, and how do we recognise consciousness.
Post reply on HN