Earlier quoted context omitted.
If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.
My layman thought about that is that, with consciousness, the medium IS the consciousness -- the actual intelligence is in the tangible material of the "circuitry" of the brain. What we call consciousness is an emergent property of an unbelievably complex organ (that we will probably never fully understand or be able to precisely model). Any models that attempt to replicate those phenomena will be of lower fidelity a…
You are the scariest monster in the woods
361–370 of 405 posts
Re: You are the scariest monster in the woods
#362Earlier quoted context omitted.
This is all very hand wavy. You don't address in the least what I've written. My criticisms stand. Otherwise... > I'm just assuming materialism, and that assumption is basically for complete lack of convincing alternatives (to me). What do you mean by "materialism"? Materialism has a precise meaning in metaphysics (briefly, it is the res extensa part of Cartesian dualism with the res cogitans lopped off). This brand…
By "materialism" I mean that human cognition is simply an emergent property of purely physical processes in (mostly) our brains. All the individual assumptions basically come down to that same point in my view. 1) Human intelligence is entirely explicable in evolutionary terms What would even be the alternative here? Evolution plots out a clear progression from something multi-cellular (obviously non-intelligent) to…
Again, this doesn't say what a "physical process" is, or what isn't a physical process. If "physical process" means "process", then the qualification is vacuous.
> All the individual assumptions basically come down to that same point in my view.
You're committing the fallacy of the undistributed middle. Just because both the brain and computing devices are physical, it doesn't follow that computers are capable of what the brain does. Substitute "computing devices" with "rocks".
> So either you need some magical mechanism that inserted "intelligence" at some point in our species recent evolutionary past, or an even wilder conspiracy theory (e.g. "some creator built us + current fauna exactly, and just made it look like evolution").
How intelligence came about is a separate subject, and I regrettably got sidetracked. It is irrelevant to the subject at hand. (I will say, at the risk of derailing the main discussion again, that we don't have an evolutionary explanation or any physical explanation of human intelligence. But this is a separate topic, as your main error is to assume that the physicality of intelligence entails that computation is the correct paradigm for explaining it.)
> Again, this is simply not an option if you stick to materialism in my view. you would need to assume some kind of bio-exclusive magic for this to work.
This is very difficult to address if you do not define your terms. I still don't know what matter is in your view and how intentionality fits into the picture. You can't just claim things without explanation, and "matter" is notoriously fuzzy. Try to get a physicist to define it and you'll see.
> Perceiving color is just how someones brain reacts to a stimulus? Why are you unhappy with that? What would you need from a satisfactory explanation?
I already explained that materialism suffers from issues like the problem of qualia. I took the time to give you the keywords to search for if you are not familiar with the philosophy of mind. In short, if mind is matter, and color doesn't exist in matter, then how can it exist in mind? (Again, this is tangential to the main problem with your argument.)
> Thinking, for us, results in and requires brain activity, and physically messing with our brains operation very clearly influences the whole spectrum of our cognitive capabilities, from the ability to perceive pain, color, motion, speech to consciousness itself.
I never said it doesn't involve physical activity. In fact, I even granted you, for the sake of argument, that it is entirely physical to show you the basic error you are making.
> If there was a link to something metaphysical in every persons brain, then I would expect at least some favorable indication before entertaining that notion, and I see none (or some plausible mechanism at the very least).
I don't think you know what metaphysics is. Metaphysics is not some kindof woo. It is the science of being and what must be true about reality for the observed world to be what and how it is in the most general sense. So, materialism is a metaphysical theory that claims that all that exists is matter, understood as extension in space (this is what "res extensa" refers to). But materialistic metaphysics is notoriously problematic, and I've given you one of the major problems it suffers from already (indeed, eliminativism was confected by some philosophers as a desperate attempt to save materialism from these paradoxes by making a practice out of denying observation in Procrustean fashion).
Re: You are the scariest monster in the woods
#363Earlier quoted context omitted.
If you make the same argument for flight it looks really weak. Cognition is (to me) not even the most impressive and out-of-reach achievement: That would be how (our) and animals bodies are self-assembling, self-repairing and self-replicating, with an impressive array of sensors and actors in a highly integrated package. I honestly believe our current technology is much closer to emulating a human brain than it is to…
> if you make the same argument for flight it looks really weak. flight is an extremely straightforward concept based in relatively simple physics where the majority of the critical, foundational ideas involved were already near-completely understood in the late 1700s. i really don't think it's fair to compare the two
Re: You are the scariest monster in the woods
#364Earlier quoted context omitted.
I think you could make AGI right now tbh. It’s not a function of intelligence. It’s just a function of stateful system mechanics. LLMs are just a big matrix. But what about a four line of code loop that looks like: ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` LLMs don’t experience continuous time and they don’t have an explicit decision making framework for having any agency…
So the current problem with a loop like that is that LLMs in their current form are subject to fixed point theorems, which are these pieces of abstract mathematics that come back when you start to get larger than some subset of your context window and the “big matrix” of the LLM is producing outputs which repeat the inputs. If you have ever had an llm enter one of these loops explicitly, it is infuriating. You can ty…
Every LLM is just a base model with a few things bolted on the top of it. And loops are extremely self-consistent. So LLMs LOVE their loops!
By the way, "no no no, that's a reasoning loop, I got to break it" is a behavior that larger models learn by themselves under enough RLVR stress. But you need a lot of RLVR to get to that point. And sometimes this generalizes to what looks like the LLM is just... getting bored by repetition of any kind. Who would have though.
Re: You are the scariest monster in the woods
#365Re: You are the scariest monster in the woods
#366Earlier quoted context omitted.
It's not an assumption, it's an extremely developed international field of tactical and strategic study that leads to these conclusions
Extremely developed thought experiment maybe. Only 2 nuclear weapons have ever been dropped. Which is why I say it's a massive assumption. You tell me. How does this escalate into a total destruction scenario? Russia uses a small nuke on a military target in the middle of nowhere Ukraine. ___________________________. Everyone is firing nukes at eachother. Fill in the blank. We are not talking about the scenario where…
While all-out international global war isn't guaranteed in this scenario, I don't see why you'd be so confident as to imply that it was very unlikely. For me, the biggest fear in terms of escalating to nuclear war is the time when some nuclear power is eventually beaten in conventional war to the point of utter desperation, and ends up going all out with nukes as a Hail Mary or a "if I can't have it, no one will" move.
Russia uses a strategic nuke in a military move in Ukraine. The rest of Europe, fearing a normalization of strategic nuke use and more widespread uses of these weapons (including outside Ukraine) begin deploying in Ukraine to help push back the Russian forces - especially since Russia showing the willingness to use nukes at all makes them and their military a lot more intimidating and urgently threatening to the rest of Europe than before. Russia perceives this deployment as a direct declaration of war from NATO and invades (one of) the Baltic states to create instability and drive the attention and manpower away from their primary front line. This leads to full war and mobilization in Europe. Russia is eventually pushed back to their original borders, but with what the situation became, Western countries are nervous that not dealing with this once and for all would just be giving Russia a timeout to regroup and re-invade with more nukes at a later point. They press on, and eventually Russia is put in a desperate situation, which leads to them using nukes more broadly against their enemies for one of the reasons I described at the start of this comment. Other nuclear states begin targeting known nuclear launch sites in Russia with strikes of their own, to cripple their launch ability. This is nuclear war.
I'm not saying this scenario is likely, but this is just one attempt at filling in the blank. If you can imagine a future - any future at all where Russia, or North Korea, or India, or Pakistan, or Israel have their existence threatened at any point in the future ever, this is when nuclear war becomes a serious possibility.
Re: You are the scariest monster in the woods
#367Earlier quoted context omitted.
It's not an assumption, it's an extremely developed international field of tactical and strategic study that leads to these conclusions
Extremely developed thought experiment maybe. Only 2 nuclear weapons have ever been dropped. Which is why I say it's a massive assumption. You tell me. How does this escalate into a total destruction scenario? Russia uses a small nuke on a military target in the middle of nowhere Ukraine. ___________________________. Everyone is firing nukes at eachother. Fill in the blank. We are not talking about the scenario where…
Let me ask you this question. Why did Russia use a small nuke on a military target in the middle of nowhere Ukraine? Because the outcome was positive for Russia... but the only way that can be true is if the cost of using a small nuke was better than the alternatives. This either means it was a demonstration / political action or... 1 or more Russian units in the Ukraine are armed with tactical nukes and it was a militarily sound option so by definition you'll see more nuke flying around at least from the Russian side when it's militarily sound. Now due to the realities of logistics that means there is capturable nuclear material on the battlefield.
If it's a demonstration/political action what do you think it was meant to accomplish? Either the consequences will be less detrimental than the military gain and so Russia can use tactical nukes and will do so if it improves the military situation... or the consequences will be at a level detrimental to Russia.
See the premise of the question is flawed in that Russia doesn't just use one Nuke in the middle of nowhere, because everyone already knows Russia has nukes. Russia is trying to demonstrate it will use them and so the actions from that are either a Ukraine alliance capitulation, Russia can continue the war just the same but with whatever ever extra political consequences of using 1 nuke, or continue the situation using Nukes.
You see the issue right? Should the Ukraine alliance surrender to a single tactical nuke when it hasn't to the threat of strategic nukes? Russia can't fire that first Nuke without being a country willing to use tactical nukes on the battlefield and what did they gain if not the use of those nukes to ensure a military victory because it hasn't ensured a diplomatic one.
So the statement becomes: Russia uses tactical nukes across the battlefield. ____. Everyone is firing nukes at eachother.
That last sentence is synonymous with "Strategic nukes being fired by ICBM" which is incredibly likely when 1 non scheduled ICBM is fired. While you're right that 1 tactical nuke in the middle of nowhere wouldn't ensure MAD, it is not a massive assumption that the realities around that 1 nuke being used would.
Re: You are the scariest monster in the woods
#368I think this is interesting and partially true: humans are scary. But it's important to remember the opposite is true as well: humans are the most cooperative species out there by a wide margin. Eusocial insects and pack animals are in a distant 2nd and 3rd place: they generally don't cooperate much past their immediate kin group. Only humans create vast networks of trade and information sharing. Only humans establis…
I'm not sure if it's your intention but this reads as a strong critique of the technolibertarianism world philosophy that dominates our industry. We lose something by replacing high-trust cooperative systems with ones that are mutually antagonistic. We fall into the bad square of the prisoners dilemma by not only not exorcising the defectors but holding them up as the highest moral good and the example to follow.
Canadians have a high-trust culture, but their stock market is historically full of scams[1], and some analysts think its causally related. (It may just be because TSXV is a wild west, or because companies would IPO on NYSE or Nasdaq if they were legit, but it could be the trust thing. Fits my narrative, anyway.)
When I look at politics, crypto rug pulls, meme stocks with P/E ratios over 200, Aum[2] and similar cults, or many other modern problems I don't see negotiations breaking down because of a lack of trust; I see a bunch of people placing far too much trust in sketchy leaders and ideas backed by scant evidence. A little skepticism would go a long way.
That's why I emphasize robust coordination: more due diligence, more transparency, more fraud detection, more skepticism, more financial literacy, more education in general. There's a cost associated with all this, sure, but it still gets you into a situation where the interaction is a coordination game[3] and the Nash equilibrium is Pareto-efficient. Thus, we fall into the "pit of success" and naturally cooperate in our own best interests.
There's nothing wrong with empathy, altruism, or charity, but they are very far from universal. You need to base your society on a firm foundation of robust coordination, and then you can have those things afterwards, as a little treat.
[1]: https://en.wikipedia.org/wiki/Vancouver_Stock_Exchange
[2]: https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack
Re: You are the scariest monster in the woods
#369I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…
If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.
An infinitely intelligent creature still has to create a standard model from scratch. We’re leaning too hard on the deductive conception of the world, when reality is, it took hundreds of thousands of years for humans as intelligent as we are to split the atom.
Re: You are the scariest monster in the woods
#370I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…
I find myself in this type of discussion with AI maximalists where they balk at me suggesting there isn’t much “I” in “AI” and they get upset that I’m not seeing how smart it is and shocked I think it’s impossible… and then they start adding all the equivocation about time horizons. I never said it wasn’t possible eventually, just not right now. If I try to pin people down to a timeline it all of a sudden becomes “surely eventually”…