Lol at George thinking _they_ are gonna let you run your own model. They don't even let me run my own adblocker!
AI 2040 and the cult of intelligence
121–130 of 280 posts
Re: AI 2040 and the cult of intelligence
#122I have some sympathy to geohot's view when it comes to pure informational chatbots. It's a first amendment issue, I'm allowed to write and read books that are useful to getting away with crimes, etc. This obviously doesn't work at all when the agents start doing real things in the real world, though. "Hey AI, I don't like my neighbor, find an exploit in the firmware for his car and make the cruise control malfunction…
"The AI" cannot "refuse" any more than a hammer can refuse. If you use a hammer to kill someone, you commit a crime and the hammer is unprosecuted. If you design a hammer to kill people and then give it away, you are partially liable for the deaths it causes. If you make a normal hammer and someone uses it to kill, you bear no blame. I don't see why these standards should change when the hammer also emits text messag…
> If you design a hammer to kill people and then give it away, you are partially liable for the deaths it causes.
This doesn't seem to be the case today (weapons manufacture).
Re: AI 2040 and the cult of intelligence
#123What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”
In fact I find it a really bad example. Yes I think your personal AI could be allowed to tell you how to cover up a murder. I'm not entirely sure about it but seems possible.
What about your personal, local AI guiding you to modifying a flu virus for maximum contagiousness and deadliness? "Sure George, here's your shopping list. It's $1500 total in equipment, do you want me to proceed with the orders?"
Re: AI 2040 and the cult of intelligence
#124Earlier quoted context omitted.
The guy was saying that AI was going to be a serious problem decades before anyone else was even considering it as a remote possibility. How is that in any way "consistently wrong with all their predictions"?
Has AI yet been a serious problem?
Yes, repeatedly, since well before this particular AI summer.
One previous attempt at AI was "expert systems", and while the term fell out of use it's functionally about the same as basically all modern business systems, and doing that wrong led to a lot of people being prosecuted for crimes they didn't commit: https://en.wikipedia.org/wiki/British_Post_Office_scandal
If you insist on learning systems, all AB testing counts, and from that we get meta making their products into hyper-stimuli for vulnerable teens: https://finance.yahoo.com/markets/stocks/articles/meta-meta-...
Even without that lawsuit, there have been various concerns that "the algorithm" (of Twitter/X, FB, Instagram, YouTube, Google, everyone) has propagandising biases that damage society. I don't know how to sift fact from politicking with that.
Re: AI 2040 and the cult of intelligence
#125If you're going to take the time to read Geohot, take it with a grain/boulder of salt: he appears to enjoy poking the bear / being provocative, sometimes outrageously. Sometimes he's writing hyperbole that he thinks is hyperbole to make a point or just enjoy taking the extreme stance. Sometimes he's dead serious. Anyway, he touches on 2 points that I think don't get enough coverage: 1: > Software didn’t eat the world…
Except paperwork is how the world turns. Say you want to throw an event. A medium sized gathering. More than a couple of friends. What happens if someone gets hurt at this event? Who's liable? Who's insured for what? It's boring and annoying, but if you throw an event without insurance and something happens, you're gonna have problems. So that paperwork needs to happen. Yes, it's not "work" the same way as actually organizing a band or being in one to play for the event is, but it's work nonetheless. AI is able to lighten the load of that very real actual work thank needs to happen in an adult world with responsibilities and liabilities. You may not respect it and think it's "bullshit", but having a plan that isn't "lol, oops" for what happens when something goes wrong, which it will, isn't bullshit, it's called being prepared and not being irresponsible. We could all just fly by the seat of our pants and run away if something bad happens, but collectively we'd be worse off for that. If I could wave a magic wand and live in a world where accidents never happen, I would, but unfortunately, I don't have one, so we're forced to live in a world of paperwork and responsibilities.
Re: AI 2040 and the cult of intelligence
#126> You cannot take over the world with tokens. I have a nice thought experiment I like to do with people when confronted with "AI can't do x". Let's go back in time. How much do we need to go for this to become true? So let's try the 2000s. Say you get a "fable/mythos/sol/gemini/kimi/glm/deepseek/whatever" in a box (and let's assume no guardrails). And you go back to the year 2005. It's "20 years ago", the world is sl…
In your hypothetical, I think it would be more apt to ask if it could do so strictly using training data pre 2005. Seems more analogous to "could it take over the world today".
No, training data has nothing to do with this. My point was that "you can't take over the world with tokens" is not a valid counterpoint in the doomer's position. Because their view is that singularity can happen. And when that happens things become murky. And 20 years of progress can go in 2 months "real time". Or two weeks. You get the point.
(I don't share their view, btw. I was just trying to say that the argument of tokens can't affect the real world is not a good one, if you steelman their maximalist position)
Re: AI 2040 and the cult of intelligence
#127Earlier quoted context omitted.
sorry, could you explain what Calvinism is for someone who (is a programmer and heard about EA and) grew up in a post-Soviet atheist metallurgy town? thanks!
Calvinism is a variant of Christianity with the following tenets ( https://en.wikipedia.org/wiki/Five_Points_of_Calvinism ): - everyone is damned and depraved - god chose some people to save, at his own whim, for no reasons related to their actions - jesus' death was a sacrifice to atone for those people, not for everyone - the chosen can't opt-out (I'm merging two into one here) I don't really see the connection to…
Something about arbitrary moralizing? Robert Louis Stevenson had some sort of lung disease in childhood and his very Calvinist nurse told him lots of stories while he was sick in bed. Then later his own short stories generally had this creepy sense that moral retribution is coming to get you from an unpredictable direction.
Re: AI 2040 and the cult of intelligence
#128I, for one, think the ChatGPT response to, "Hey, I just killed my wife ..." is not bad and preferable to it helping him.
Re: AI 2040 and the cult of intelligence
#129Earlier quoted context omitted.
> Seems like being directionally right early that AI would be a Big Deal and scary should count for something? If I scream that a vicious beast is going to destroy the world, I don't get credit for being 'directionally right' if a squirrel eats a hazelnut. AI being a big deal is a long, long way from the full beliefs Yudkowksy promotes, and there are many people who predicted AI's significance who didn't also believe…
This doesn’t track though. Just recently AI became “powerful enough” to warrant export controls from the US and maybe soon China. It’s more he screamed that bacteria is gonna do evolution faster than any before it and turn into a beast that destroys the world, and now that bacteria has turned into a squirrel so far. He’s been very right so far and if that squirrel keeps sizing up you’d want to give him credit.
Just recently AI became “powerful enough” to warrant export controls from the US and maybe soon China.
Someone makes a made up claim, second party acts on made up claim, third party takes second party’s actions a proof of claim.
Reminds me of the “they’re eating pets” thing a bit.
Re: AI 2040 and the cult of intelligence
#130> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. I really think we need to stop giving credence to people who have 1) Been consistently wrong with all their predictions 2) Demonstrated an endless spiteful cynicism Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk…
The cynicism and misanthropy is nonsense and a direct result of the internet/social media landscape. It's BS. In the current media ecosystem, what gets you "social cache" or whatever? Let's call them "cool points." Well, right now, the current metric is basic "likes" or "retweens" or "upvotes" or whatever. And to get those, you have to make a claim and then that claim has to be evaluated by others. But the evaluation…
Your theory is pretty silly. If the outcome is doom, nobody cares whether you were right, because humans are dead or irrelevant. The asymmetry in the reward function isn't there.
In general the rewards for public doomerism seem low, except if you think being public about it can shift the outcomes.