Was expecting a little more analysis of or at least information about the content of the paper itself.
I updated the post with a a link to counter-argument from Sabine Hossenfelder, the arguments from Zvi and three points from my side.
Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
61–70 of 70 posts
Re: Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
#62Few times seen a blog post touching on so many subjects, without actually saying anything. Is there an argument, point, story, poem, news, analysis or something being made here?
I updated the post with a a link to counter-argument from Sabine Hossenfelder, the arguments from Zvi and three points from my side.
Re: Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
#63It starts with the sentence "You can see the future first in San Francisco" and while the following paragraphs make a good, positive upbeat point, reading it out of context I cannot help thinking about homeless people in the streets and a pretty dark and dystopian future. It's interesting how a place can be so radically different things at the same time.
> It starts with the sentence
Reminds me of William Gibson's opening line in "Neuromancer":
The sky above the port was the color of television, tuned to a dead channel.
Re: Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
#64My question after reading the paper is: if he believe with conviction everything he wrote, why did he recently become an investor instead of either working in policy or working on the tech itself? If the tech is going to transform humanity into a new economic paradigm and give the country controlling it a decisive advantage to be the sole superpower, being an investor is either futile or meaningless. Even doing nothi…
> why did he recently become an investor instead of either working in policy or working on the tech itself?
Even he needs a nest egg.
Re: Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
#65The tone of Aschenbrenner's paper is odd. It's somewhere between his wet dreams and his nightmares. In fairness he seems to be aware of that but it's like someone grinning reflexively while they tell you bad news. He's convinced and it's going to be terrible and it's going to be important and he's going to be part of it. I guess for sure in 5 years it will be much clearer.
I'm familiar with this tone from my LessWrong lurker days. It comes from wrestling with the idea that there is no scalable way to actually stop AGI from coming about, so you just have to grit your teeth and hope for the best.
I mean, the numbers for the money in the paper in terms of investment mean that spending is on track for what he’s talking about. 4 to 7 more years is a lot but it’s not entirely outside the realm of possibility.
My understanding of the models and linear algebra combined with my experience of their performance improvements make me think he’s likely right. We have some people wanting to pooh-pooh these machines. They will point to current limitations of the models and weak spots as though the underlying mode of the design rather than the current implementation is the limiting factor. They act as though real intelligence is always right.
I have been wrong before but I am pretty convinced. Everyone’s arguments against it happening is that it hasn’t yet happened.
No one knows what’s on the other side of AI being a better AI researcher than humans.
Re: Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
#66Earlier quoted context omitted.
My point is that it is meaningless to make a lot of money for his scenario
Do you think all previous wealth will simply be 'reset' in some way, come the AI rapture?
Yeah, that seems like a weird assumption to make.
I guess one could make the argument that wealth will be useless in a post-scarcity scenario, so why not spend for today. But, and this might just be me speaking, I sleep a lot better at night now having some nonzero amount of money invested and going to work for me one way or another.
Re: Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
#67Earlier quoted context omitted.
I'm familiar with this tone from my LessWrong lurker days. It comes from wrestling with the idea that there is no scalable way to actually stop AGI from coming about, so you just have to grit your teeth and hope for the best.
I mean, the numbers for the money in the paper in terms of investment mean that spending is on track for what he’s talking about. 4 to 7 more years is a lot but it’s not entirely outside the realm of possibility. My understanding of the models and linear algebra combined with my experience of their performance improvements make me think he’s likely right. We have some people wanting to pooh-pooh these machines. They…
Agreed with you that (a) AGI is a real, transformative, and possibly calamitous event, and (b) with the amount of money getting pumped into it we will almost certainly get there.
However, I have a personal hobby horse here, which is reminding people that if we shut the flow of investor money off using the right economic policies [1], we can almost certainly just ... stop the research outright, before it becomes an issue.
[1]: [url-redacted]
Re: Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
#68So if we supposedly already have passed the point where LLMs are at the level of high schoolers, how comes that almost no tasks or jobs that high-schoolers are capable of doing have been replaced by LLMs? Neither will more advanced work be replaced by 2027. End of story.
I'm not on board the hype train either, but I don't think that's the right way to look at it. High schoolers are already not widely employed for knowledge-based work. It's almost all physical labor. The few knowledge jobs among my peers in high school were tutoring and call centers, both of which are arguably replaceable with LLMs today. But "replaced" isn't really the right frame either. In my very physically-constr…
I did not choose this framing myself, I was responding to the paper which claimed that these jobs will be fully replaced by 2027.
And regarding your other point:
> both of which are arguably replaceable with LLMs today
Arguably, but they are not. That‘s my point.
Re: Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
#69There's a Sabine Hossenfelder youtube just out that is more interesting than the linked article about it. "Is the Intelligence-Explosion Near? A Reality Check." https://youtu.be/xm1B3Y3ypoE
Re: Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?
#70Earlier quoted context omitted.
I mean, the numbers for the money in the paper in terms of investment mean that spending is on track for what he’s talking about. 4 to 7 more years is a lot but it’s not entirely outside the realm of possibility. My understanding of the models and linear algebra combined with my experience of their performance improvements make me think he’s likely right. We have some people wanting to pooh-pooh these machines. They…
Agreed with you that (a) AGI is a real, transformative, and possibly calamitous event, and (b) with the amount of money getting pumped into it we will almost certainly get there. However, I have a personal hobby horse here, which is reminding people that if we shut the flow of investor money off using the right economic policies [1], we can almost certainly just ... stop the research outright, before it becomes an is…
True but my view on that is that governments don't want anyone else to be the first with AI AGI... and therefore they will strive to be the first. Kinda like how a big motivation for the atomic bomb was the threat of Germany developing the bomb.
And the amount of capital means that these companies could pay a lot more for labour than they are paying now for the sorts of labour we would need to deprive them of. It's an interesting idea though.