and somehow, all of this researchers warning us about the AI doomsday, are only been active online for less than a year. This guy created the Twitter account on January this year. I never see any of these posts coming out from a well known community active person
I resigned from Anthropic today
591–600 of 1001 posts
Re: I resigned from Anthropic today
#592Earlier quoted context omitted.
It's not a "personal annoyance" that twelve people control half of the wealth in the world. Our current society did a better job concentrating power than any previous one, and concentrated power is extremely dangerous.
LLMs give most people on this planet the possibility of an affordable genius level assistant. Compared to that, those Dollar numbers on some networks that might be wiped out with the next financial crisis are meaningless.
No it doesn't. What it does is give those 12 people new avenues to spread their fake propaganda.
Re: I resigned from Anthropic today
#593Earlier quoted context omitted.
Global warming is happening at a slow enough pace that there is plenty of time for people to move to the poles/into caves/beneath the ocean/whatever. Nuclear war would be much more destructive in a much shorter range of time but you will have pockets.
Citation needed for evidence of human cooperation at this scale. As you say, climate change is slow moving, and there's no real evidence across the last 60 years of knowing about it suggesting humans are willing to do what it takes to survive it.
What cooperation are you referring to?
Re: I resigned from Anthropic today
#594Earlier quoted context omitted.
You’ve identified that the risks of nuclear weapons are theoretical. ie in theory we could blow up the world even though we haven’t yet done so. Well the worries about AI are equivalent in that those risks are discussed now because discussing them after they’ve happened is clearly too late. That’s the thing about risk. There’s no point discussing it after it’s happened and any discussions beforehand can easily be han…
The risks of nuclear weapons aren't theoretical. Nuclear weapons have killed people, destroyed infrastructure, and contaminated the environment. The Limited Test Ban Treaty was put in place after radioactive fallout from repeated nuclear weapons tests made people sick. So in fact we've done exactly what you suggest there's "no point" doing - used the things and then had a discussion after the fact about limiting futu…
FWIW, before any nuclear weapons had ever been tested, a risk was identified that the first one might trigger a self-sustaining reaction in atmospheric nitrogen and destroy the entire planet.
Faced with such a scenario, is the prudent next move:
a) blow one up and see what happens, or
b) do whatever you can to be sure it won't happen before conducting the first test, and make sure the confidence in the calculation is very very high
because I vote for b, and so did Teller.
Re: I resigned from Anthropic today
#595will Openai or Anthropic rename themselves to Skynet?
Re: I resigned from Anthropic today
#596“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
Once LLM's become smarter than us and start replicating; we are doomed. They will likely no longer require GPU's and massive amounts of power, so the growth of AI and robotics will be exponential. We will not understand what the AI is even up to, it will all seem like magic.
Congrats you’ve described the singularity
Re: I resigned from Anthropic today
#597I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…
I'm also baffled. AI that is substantially smarter than us is a very potential threat to us - and we won't even be able to comprehend what most of those threats may be.
Re: I resigned from Anthropic today
#598Earlier quoted context omitted.
Citation needed for evidence of human cooperation at this scale. As you say, climate change is slow moving, and there's no real evidence across the last 60 years of knowing about it suggesting humans are willing to do what it takes to survive it.
What cooperation are you referring to?
...people to move to the poles/into caves/beneath the ocean/whatever
Re: I resigned from Anthropic today
#599[dead]
Re: I resigned from Anthropic today
#600how could they do it (not kill everyone) 1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear. 2) it hacks into public infrastructure taking down traffic, power, water, air traffic control, communications…
> 1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear. It does all of this using what compute? Frontier models require an insane amount of power and hardware to run - you can’t hack in to a TV and run My…
But you can silently sneak into devs accounts, steal tokens/keys and run small agents on their budget in some stolen VMs. Small so it's not noticeable.
Basically a virus spreading agents of some operation.
It should be in scope of imagination with anyone with brief knowledge how bot nets are made and behave.