911–920 of 947 posts
> One way that thinking for yourself goes wrong is that you realize your society is wrong about something, don’t realize that you can’t outperform it, and wind up even wronger. I've been there myself. > And without the steadying influence of some kind of external goal you either achieve or don’t achieve, your beliefs can get arbitrarily disconnected from reality — which is very dangerous if you’re going to act on the…
stimulants
Earlier quoted context omitted.
I think it's a meaningful distinction- most rationalists aren't running murder cults.
Funny how the comment making a factual correction gets downvoted. For me, that is the crucial information in the article: Yes, multiple people have succeeded to create a cult within the rationality community, but it always involved isolating their victims from the rest of the rationality community. (Now that we see the pattern, could it possibly help us defend against this?)
Earlier quoted context omitted.
Yes, I agree they are saying it is likely going to kill us all. My interpretation is consistent with that, and so is yours. The difference is in why/how it will kill us; you sound to me like you think the rationalist position is that from intelligence follows malice, and therefore it will kill us. I think that's a wrong interpretation of their views.
Well then, instead of just telling me I'm wrong why don't you tell me why I'm wrong.
Earlier quoted context omitted.
Funny how the comment making a factual correction gets downvoted. For me, that is the crucial information in the article: Yes, multiple people have succeeded to create a cult within the rationality community, but it always involved isolating their victims from the rest of the rationality community. (Now that we see the pattern, could it possibly help us defend against this?)
If it took you this long to see a pattern that anyone with any experience at all can trivially recognize, why should anyone trust you to defend against anything? You are plainly incompetent.
> This article attempts to establish disjoint categories "good rationalist" and "cultist."
...that you have denied in another comment in the same thread. I guess that makes two of us plainly incompetent.
Earlier quoted context omitted.
Ah, I see the confusion now. It is you that actually does not know what the word means or how it was used in this context. Several hundred other people seemed to have avoided this confusion though, and you still haven’t said how it was used wrong, so I can safely dismiss you as a troll. hint: it was once axiomatic that the sun revolved around the earth. Hope that helps!
Quantity of idiots agreeing with someone does not make someone right. Keep being delusional.
Earlier quoted context omitted.
If it took you this long to see a pattern that anyone with any experience at all can trivially recognize, why should anyone trust you to defend against anything? You are plainly incompetent.
Yes, I am definitely plainly incompetent for not noticing a trivially recognizable pattern... > This article attempts to establish disjoint categories "good rationalist" and "cultist." ...that you have denied in another comment in the same thread. I guess that makes two of us plainly incompetent.
Earlier quoted context omitted.
People reading the book and being into it and telling other people. It’s also a hard book to read so it may be smart kids trying to signal being smart.
Fountainhead is written at the 7th grade reading level. Its Lexile level is 780L. It's long and that's about it. By comparison, 1984 is 1090L.
Earlier quoted context omitted.
I think you'll indeed find, if you were to seek out the relevant literature, that those claims are more or less true, or at least, are the currently best-supported interpretation available. So I don't think they're assumptions so much as simply current state of the science on the matter, and therefore widely accepted among those who for whatever reason have looked into it (or, more likely, inherited the information f…
My general disagreements with those axioms from my reading of the literature are around the concepts of immutability and of the belief in the almost entirely biological factor, which I don't think is well supported by current research in genetics, but that may change in the future. I think primarily I disagree about the effect sizes and composition of factors with many who hold these beliefs. I do agree with you in t…
They'd agree it's largely stable over life, after whatever childhood environmental experiences shape that "non-shared environment" bit.
This is the current state of knowledge in the field as far as I know - IQ is about half genetic, and fairly immutable after adulthood. I think you'll find the current state of the field supports this.
Earlier quoted context omitted.
On the other hand, I'm baffled to encounter recursive self-improvement being discussed as something not only weird to expect, but as damning evidence of sloppy thinking by those who speculate about it. We have an existence proof for intelligence that can improve AI: humans. If AI ever gets to human-level intelligence, it would be quite strange if it couldn't improve itself. Are people really that sceptical that AI wi…
Consider that even the named phenomenon is sloppy: "recursive self improvement" does not imply "self improvement without bounds". This is the "what if you hit diminishing returns and never get past it" claim. Absolutely no justification for the jump, ever, among AI boosters. > If AI ever gets to human-level intelligence This picture of intelligence as a numerical scale that you just go up or down, with ants at the bo…
Obviously not, but thinking that the bounds are going to lie in between where AI intelligence is now and human intelligence I think is unwarranted - as mentioned, humans are unlikely to be the peak of what's possible since evolution did not optimise us for intelligence alone.
If you think the recursive self-improvement people are arguing for improvement without bounds, I think you're simply mistaken, and it seems like you have not made a good faith effort to understand their view.
AI only needs to be somewhat smarter than humans to be very powerful, the only arguments worth having IMHO are over whether recursive self-improvement will lead to AI being a head above humans or not. Diminishing returns will happen at some point (in the extreme due to fundamental physics, if nothing sooner), but whether it happens in time to prevent AI from becoming meaningfully more powerful than humans is the relevant question.
> we do not have a definition of intelligence
This strikes me as an unserious argument to make. Some animals are clearly more intelligent than others, whether you use a shaky definition or not. Pick whatever metric of performance on intellectual tasks you like, there is such a thing as human-level performance, and humans and AIs can be compared. You can't even make your subsequent arguments about AI performance being made worse by various factors unless you acknowledge such performance is measuring something meaningful. You can't even argue against recursive self-improvement if you reject that there is anything measurable that can be improved. I think you should retract this point as it prevents you making your own arguments.
> There is not even any evidence that LLMs will continue increasing as techniques improve, except in the tautological sense that if LLMs don't appear to resemble humans more closely we will call the technique a failure.
I'm pretty confused by this claim - whatever our difficulties defining intelligence, "resembling humans" is not it. Do you not believe there are tasks on which performance can be objectively graded beyond similarity to humans? I think it's quite easy to define tasks that we can judge the success of without being able to do it ourselves. If AI solves all the Millennium Prize Problems, that would be amazing! I don't need to have resolved all issues with a definition of intelligence to be impressed.
Anyway, is there really no evidence? AI having improved so far is not any evidence that it might continue, even a little bit? Are we really helpless to predict whether there will be any better chatbots released in the remainder of this year than we already have?
I do not think we are that helpless - if you entirely reject past trends as an indicator of future trends, and treat them as literally zero evidence at all, then this is simply faulty reasoning. Past trends are not a guarantee of future trends, but neither are they zero evidence. They are a nonzero medium amount of evidence, the strength of which depends on how long the trends have been going on and how well we understand the fundamentals driving them.
> thinking clearly is about the reasoning, not the conclusion.
And I think we have good arguments! You seem to have strong priors that the default is that machines can't reach human intelligence/performance or beyond, and you really need convincing otherwise. I think the fact that we have an existence proof in humans of human intelligence and an algorithm to get there proves it's possible. And I consider it quite unlikely that humans are the peak of intelligence/performance-on-whatever-metrics that is possible given it's now what we were optimised for specifically.
All your arguments about why progress might slow or stop short of superhuman-levels are legitimate and can't be ruled out, and yet these things have not been limiting factors so far despite that they would have been equally valid to make these arguments any time in the past few years.
> no legitimate argument has been presented that implies the conclusion
I mean it's probabilistic, right? I'm expecting something like an 85% chance of AGI before 2040. I don't think it's guaranteed, but when you look at progress so far, and that nature gives us proof (in the form of the human brain) that it's not impossible in any fundamental way, I think that's reasonable. Reasonability arguments and extrapolations are all we have, we can't imply anything definitively.
You think what probability?
Interested in a bet?
Earlier quoted context omitted.
On the other hand, I'm baffled to encounter recursive self-improvement being discussed as something not only weird to expect, but as damning evidence of sloppy thinking by those who speculate about it. We have an existence proof for intelligence that can improve AI: humans. If AI ever gets to human-level intelligence, it would be quite strange if it couldn't improve itself. Are people really that sceptical that AI wi…
> We have an existence proof for intelligence that can improve AI: humans. I don't understand what you mean by this. The human brain has not meaningfully changed, biologically, in the past 40,000 years. We, collectively, have built a larger base of knowledge and learned to cooperate effectively enough to make large changes to our environment. But that is not the same thing as recursive self-improvement. No one has be…
Yes it took billions of years all said and done, but it shows that there are no fundamental limits that prevent this level of intelligence. It even proves it can in principle be done with a few tens of watts a certain approximate amount of computational power.
Some used to think the first AIs would be brain uploads, for this reason. They thought we'd have the computing power and scanning techniques to scan and simulate all the neurons of a human brain before inventing any other architecture capable of coming close to the same level of intelligence. That now looks to be less likely.
Current state of the art AI still operate with less computational power than the human brain, and they are far less efficient at learning that humans are (there is a sense in which a human intelligence takes a merely years to develop - i.e. childhood - rather than billions, this is also a relevant comparison to make). Humans can learn from far fewer examples than current AI can.
So we've got some catching up to do - but humans prove it's possible.