AI 2040 and the cult of intelligence

This part is a parallel line of values, and if we were bound together in the same community and had to set rules, I would have debated it. But since we both know we come from different regions and backgrounds, we know our agreement will never converge.

That is why I will only agree with your last opinion and will not discuss the parts you mentioned earlier. I agree with your last point that it "concentrates power in the hands of those who already have a lot of it"

Just strike that weird nugget out, I guess?

In the current media ecosystem, what gets you "social cache" or whatever? Let's call them "cool points." Well, right now, the current metric is basic "likes" or "retweens" or "upvotes" or whatever. And to get those, you have to make a claim and then that claim has to be evaluated by others. But the evaluation by others is not really reality, it's just what others think, and it's based on outcomes.

This gives us 4 possible prediction-outcome scores.

You predict DOOOOOOOM!, and the outcome is DOOOOOOM! - in this case you look like a genius because you predicted doom

You predict DOOOOOOM!, and the outcome is "ok, or even great!" - nobody cares you got it wrong because things are fine

You predict UTOPIA!, and the outcome is "ok, or even great!" - people say, "wow that was cool, you got it right" but its not that great of a reward from your social group (the internet) because things were fine

You predict UTOPIA!, and it turns out DOOOOOM! - you look like a moron.

Now looking like a moron is is wayyyyyy worse than what happens if you get it right. Indeed, unless you are really damn sure of utopia, you have a social incentive to predict DOOOOOM! that's greater than the alternative. You can predict DOOOOOM! 100 times and get it wrong 99 of them and nobody cares because that one time you get it right you look like a genius. It's a huge huge problem in the current media landscape and it's why everything is killer robots right now and not "hey we could build a star trek utopia!" It's effectively a selfish NIMBY-ist view of the world in a way, "why build anything, it's just going to ruin everything!" It's a pessimism that literally desires the end of humanity to "prevent the likely future sufferings." It's a simplistic view of a complicated world, and the rewards that go to "DOOOOM!" don't match much with reality.

That's why the internet is so doom pilled, that's why everyone is a cynic, that's why everyone is kind of an asocial asshole about new ideas on the internet and even in your friend space.

You guys know you can just "do stuff?" I mean, yeah it takes money and time, but you can allocate that - especially now that you don't have to grind out css for hours and hours. But you can just do things - whatever you want. You've always been able to, but it just got a lot easier in many many domains, and it's going to get even easier than that.

People are going to scream slop and "that sucks" until the cows come home, because that's what is rewarded on the internet these days. Meanwhile, as people we MUST believe that we still have some agency to do stuff, then go and actually do stuff while the haters are sitting on their couches shellacking each other with misanthropy.

And don't even get me started on the loudest voices in the ecosystem right now.

Like, Yudkowsky is a nobody. Not to be a dick, but he's a high school dropout forum guy who got lucky. I'm not saying he's always wrong, or even that I oppose autodidact stuff - indeed I'm always self teaching. But this guy never took a ML class in his life, I doubt he can solve an integral un-aided, I imagine some people here have seen this: https://www.reddit.com/r/badmathematics/comments/2bazyc/comm...

And we're all deferring to that guy like he knows wtf he's talking about? It's the AI equivalent of the self-educated anti-vax mom at the height of the pandemic.

And he's not alone either, there are a lot of people who are uninformed, unknowledgeable, and confidently wrong on the internet desperately trying to peel off your attention for an instant so you can look at their site, their ads, and their content.

The same sorts of things can be said for the Zitrons of the world. He's a hell of an entertainer, and really really funny, but he's consistently wrong on AI stuff. He's got a BA (with honours I guess?) in media something or other. And again, he's really funny, but does that make him qualified to pontificate on matters of AI policy or economics or even to make statements about the technology etc?

Regardless, the misanthropy, the cynicism, and the misinformation that's being shouted about is because being a cynic is more "profitable" than being an optimist when you measure your success more by how many people agree with you rather than actually being right and being able to do things.

Anyway, he touches on 2 points that I think don't get enough coverage:

1:

> Software didn’t eat the world, it largely removed one layer of friction then reintroduced it for the benefit of a few tech companies.

USA-style capitalism has shown itself to be fantastic at seeking profit. Unfortunately, it turns out that in the context of long standing regulation, it is far far more profitable to exploit regulatory loopholes and user/human irrationality/weaknesses than to increase productivity.

The upshot of this is that the arc of most companies is to first serve the user and then leverage that relationship to exploit market and consumer weakness. So you get UI dark patterns, non-colluding oligopolistic price manipulation, outsourcing of costs to public entities, etc. while the core value proposition erodes as the company strategy turns ever more into making profit by legal anti-competitive tactics.

This company arc is so prevalent it's boring: it's a reflection of the system and it's not Facebook or Google or [insert whatever], it's what the system allows and therefore incentivizes.

2:

> No matter how high quality your tokens are, they cannot turn lead into gold.... AI 2040 includes this picture of a datacenter in the ocean. Just like vaporware, you can generate a picture easily. But in reality, you have to deal with supply chains.

"Bullshit jobs" took over the zeitgeist for a turn awhile ago. So many jobs are so far removed from actually moving physical things around. It may be the case that AI will just accelerate how quickly companies generate reports about reports about possible strategies to address potential futures and pay employees to do it.

It seems that AI mostly accelerates paperwork, not production or real service work that tangibly improves real world outcomes. It's all: faster images, more and faster powerpoints, more accurate dictation to satisfy insurance requirements (that is it's own regulatory capture), etc. etc. There are exceptions, but they seem to be few.