101–110 of 113 posts
Cosmos and Culture, chapter 7: Dangerous Memes by Susan Blackmore.
Ms. Blackmore speculates about what we will find should we venture out into the cosmos. She constructs her speculations around a theory of memetics that a dangerous meme could end our civilization and leave only bones for space explorers to find.
Earlier quoted context omitted.
a) You can verify that a solution is valid in polynomial time, but you can't verify whether or to what extent it's optimal. But even if this weren't the case... b) The solution you're talking about is an update to the network. It's buried back in the network's construction, not directly visible in the network itself. Model "blame" is a thing, but not heavily researched or at all cheap, computationally. That said, bti…
True, there are lots of cases where an approximation can create provably good answers question. But they usually require things like having probabilities bounded away from 1 and 0. Unfortunately in the real world we actually are certain about lots of things. And when you add data, we tend to be more certain of previously uncertain things. Therefore we wind up with self-referential networks of beliefs that reinforce e…
Of course you actually could prove better bounds for an approximation algorithm given a specific universe of inputs. Your 'probabilities bounded away from 1 and 0' sound like an example of that (presumably the 'special cases' from the Bayesian network reference, but I only read the abstract). What I see more often are empirical studies showing, say, > 98% optimality for some kind of max cover approach to a specific problem, even though the guarantee in general is < 64%.
Earlier quoted context omitted.
Sure, have a different prior if you think those assumptions are relevant. i don't P(what if we're first?) ~ 0 P(or FTL travel isn't possible in our universe?) ~ 1
what's your basis for these assumptions?
Special relativity.
The nonexistence of grey goo (von neumann probes) is strong prior for safe agi. AI xrisk is woo. Paperclip maximizers are p-zombies. They can't exist. Chicken littles see apocalypses on every horizon even though they dont understand the technology at all. "I can imagine this destroying the world" is their justification. Even though their "imagination" is 16x16 greyscale.
Paperclip maximizers really have nothing to do with p zombies. A paperclip maximiser is simply an AI with an unconstrained goal that has unintended and bad consequences for us when taken to an extreme. It does not need to be something that could function exactly like a human without having consciousness.
Earlier quoted context omitted.
So much of this angst about AGI boils down to: “What if we can’t enslave our new God?” It’s an insane perspective, advanced by people who have no clue how cowardly and arrogant they come across.
No, it's "we won't be able to enslave our new god". The point is to delay creating it as long as possible.
Earlier quoted context omitted.
Paperclip maximizers really have nothing to do with p zombies. A paperclip maximiser is simply an AI with an unconstrained goal that has unintended and bad consequences for us when taken to an extreme. It does not need to be something that could function exactly like a human without having consciousness.
That isn't what I said and I dont think you refuted my actual statement. PMs and PZs are incoherent by construction.
I can see an argument that PZs are incoherent by construction, but I'm not sure why a PM is. Can you explain why you think that is so?
Earlier quoted context omitted.
> Human ethical norms are highly complex, and are the result of a long evolutionary history that will not be shared by any AI. The chances of any arbitrary set of ethical values being compatible with human life is very low. If AI is trained by a huge corpus of human language, it may very well share our norms/values.
Our norms and values include that we treat sentient creatures that we deem inferior as if they had no moral value. So that's not very comforting tbh.
There are about 1.2 billion hindus, and a lot of them treat cows as "sacred". Which in practice means they make sure not to hit them with cars, and just let them be. If a superhuman AI would treat us like that, that's a pretty okay scenario compared to the extinction-level ones.
One thing I've frequently noticed in the rationalist community is the belief that if we all just reason hard enough, we'll reach the same conclusions. And that disagreement just means that one side is "wrong" and that therefore more debate is needed. This seems to be connected to the belief that AI will naturally just over-optimize to turn us all into paper clips. Implicit in this belief, it seems, is that there aren…
I find it hilarious that rationalists have failed to notice or realize the consequences of the fact that approximating an update to a Bayesian network, even to getting an approximate probability answer that is within 49% of the real one, is NP hard. The consequence is that for any moderately complex set of beliefs, it is computationally impossible for us to reason hard enough about any particular observation to corre…
Yes, in general. But the usual limits of "bounded rationality" make that result basically irrelevant. Most people don't have a myriad strong beliefs.
The problem is more like "the 10 commandments are inconsistent" and not that "Rawls' reflective equilibrium might not converge".
Earlier quoted context omitted.
> societies have many mechanisms to reduce the amount of ethical pluralism such as laws, conventions & customs, peer pressure, religions and so on Constrain, yes. The same way we would seek to constrain a paperclip-maximising LLM.
Any AI smart enough to become a classical paperclip maximizer is smart enough to hide its abilities and intentions until humans no longer have the ability to constrain it.