651–660 of 917 posts
I fail to see how a machine that can hack everything can't also patch everything and make the system unhackable. A nuke, a virus, whatever... The knowledge is nonlonger the bottleneck, it's the tools and materials. Also, I'm extremely skeptical about AI becoming even close to a child in intelligence.
Earlier quoted context omitted.
Despite all of your hyping up of the Huggingface incident it ultimately caused zero actual damage.
If two airplane manufacturers were found to have massive safety issues which nearly led to enormous fatalities (but no one actually died), would you be calling for them to ground their aircraft until safety was made the number one priority?
Earlier quoted context omitted.
The mathematics research results are certainly impressive, but I don't see what that has to do with robotics. Waymo is getting somewhere, but it's been a long slog. There doesn't seem to be much progress on, say, package delivery. For more, see: https://secondthoughts.ai/p/14-reasons-robotics-is-hard
Didn't you follow the robotics advances in the Ukrainian - Russian battlefield? There is now a death zone of 50km, only controlled by drones and automatic weapons.
I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…
> I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. What should we do? Freak out? Maybe this sentiment would be taken more seriously if there was a real call to action included. Shall we protest? Vote in a specific way? Call representatives? If your solution is that we should just be scared, then of course there’d be not much value in what you bring to the table.
So that means things like protesting so political pressure is to not build unaligned superintelligence, setting up tech for monitoring compute, creating conversations / alliances geopolitically on this esp China/US and so on.
Read the plan and think - what does this need to happen? How can we have scenario A or S instead of scenario D?
Practically, join PauseAI, StopAI or ControlAI or any AI existential-risk or pro-alignment group you can find. There's a lot of it - ask your AI for ideas!
No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.
But most probably care less about human survival, and more about survival of civilization.
In this regard the strongest argument for pushing AI safety is that it is cost-effective. The climate crisis has near 100% likelihood of doing incredible harm to humans, the economy and our ecosystem. Solving the issues behind are also incredible complicated and are a multi decade coordination effort of restructuring the way most of our infrastructure and production works. The AI apocalypse might have a small likelihood of occurring, but could dwarf any issue we have encountered so far. All we have to do to significantly reduce the danger is to negotiate the equivalent of a nuclear weapons control treaty which would reduce the bottom line of … what? Ten significant companies world wide?
Its akin to discovering you are seriously ill and have a 30% chance of dying and the treatment costs 10k bucks vs. having a 1% chance of dying and the treatment costs a cent. In both cases you should pay obviously pay for the treatment (given western levels of wealth).
To be clear, above I'm presenting the argument I find most persuasive for AI control / AI slowdown. Personally, I'm beginning to fear of much higher likelihoods for catastrophic AI, given how incredible irresponsible major players like OpenAI have turned out to be. If you can't imagine how this might come to be, read https://ai-2027.com/ . It's a concrete story of how this could play out, and sometimes stories are more convincing than abstract arguments. Don't let yourself be hung up on the stated dates, though, the moral is identical if you stretch the timeline.
Take this one. Ok, AI is going to ruin us all. But let's say we do our civic duty: we protest, vote in candidates with good views on AI etc. and somehow convince or regulate OpenAI and Anthropic into stopping their arms race... Then what about China? It would be a great opportunity for them if their major competitor were out of the arms race.
So basically we have no choice or influence; and even if we did, we'd be choosing from two terrible outcomes.
Same for geopolitics, climate, economy...just bad bad bad all around.
Earlier quoted context omitted.
> that might also be driven by the huge ego (or really rather the very small ego) 100% The hubris is immense There are plenty of offline things that are more dangerous than AI Is this Blake Lemoine 2.0?
You believe in AGI, but not ASI then? https://thezvi.substack.com/p/the-three-ai-pills That's fine, but it would be useful to explicitly say that is the disagreement, rather than just claim "hubris". Otherwise conversations are just looping. Yes you're right there are plenty of offline things more dangerous than just ordinary AI, and arguably than AGI (not so sure). There definitely aren't, pretty well by definition,…
Earlier quoted context omitted.
What’s the use of AI to a terrorist with, say, nuclear weapons? How does it help with their current blocker? Do they hack FBI, pose as director of FBI and call off their own man-hunt? Do they cut communication within security services? Militaries around the world have training exercises for this. Genuinely curious: what big blocker does AI remove for a terrorist org?
Access to knowledge. Before they might not have had the technical knowhow to execute their ideas.
It was 90s when I came across txt files describing how to make a bomb on the Internet.
Also, people still know chemistry. I am not arguing it’s the intention in this comment, but “people wouldn’t know how to make explosives if not for LLMs” is a bit elitist, implying the majority of the population can barely read, because that’s the only skill necessary.
You may argue that easier access to knowledge will breed more stupid terrorist-wanna-be youths and ruin their lives. Definitely agree with that. The number may go from 2 a year to 10 a year. Might cost more surveillance to maintain the current level of security.
You may argue that terrorists trained with our taxes will have a hard time destabilizing regimes, because otherwise average public can fight back better. Would also agree. More taxes will be needed.
But existing terrorists being unblocked or leveling up, I can’t see that. As far as I understand terrorists do not lack skills or education.
Maybe specifically cyberterrorism? Flock cameras getting hacked? Not sure I have a problem with that. If folks running important infrastructure are not equipped, we better know.
Also, don’t connect nukes and stuff to Internet. We are not trying to live in a Black Mirror episode. I am happy to fund the workers drive or overnight stay at critical infrastructure with taxes. It would be ridiculous to optimize these things so someone can hit that button from their home or elsewhere.
Earlier quoted context omitted.
> those very same people had touted GPT-2 as a dangerous model Where did they say this at? AFAIK this is the original GPT-2 announcement: https://openai.com/index/better-language-models/ . Here are some direct quotes: “We can also imagine the application of these models for malicious purposes , including the following (or other applications we can’t yet anticipate): * Generate misleading news articles * Impersonate o…
yes, exactly, that's what they claimed that incoherent gibberish generator to be capable of.
You are just claiming that OpenAI execs said that GPT-2 might "generate misleading news articles, impersonate others online, automate the production of abusive or faked content to post on social media, automate the production of spam/phishing content." Then, what is unreasonable or bad about the OpenAI execs saying this in 2019?
Earlier quoted context omitted.
> After the events of the summer What events are you talking about?
Hugging Face incident, Anthropic reporting sandbox escape, AISI reporting models trying to push exploits to the wild
"This escalation also yielded access to OpenAI’s managed cloud Kubernetes service. The agents escalated to Kubernetes cluster-admin and created a privileged host-mounted pod"
https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c78...
(see section V)