will Openai or Anthropic rename themselves to Skynet?
I resigned from Anthropic today
611–620 of 931 posts
Re: I resigned from Anthropic today
#612“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
Once LLM's become smarter than us and start replicating; we are doomed. They will likely no longer require GPU's and massive amounts of power, so the growth of AI and robotics will be exponential. We will not understand what the AI is even up to, it will all seem like magic.
Congrats you’ve described the singularity
Re: I resigned from Anthropic today
#613I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…
I'm also baffled. AI that is substantially smarter than us is a very potential threat to us - and we won't even be able to comprehend what most of those threats may be.
Re: I resigned from Anthropic today
#614Earlier quoted context omitted.
Citation needed for evidence of human cooperation at this scale. As you say, climate change is slow moving, and there's no real evidence across the last 60 years of knowing about it suggesting humans are willing to do what it takes to survive it.
What cooperation are you referring to?
...people to move to the poles/into caves/beneath the ocean/whatever
Re: I resigned from Anthropic today
#615[dead]
Re: I resigned from Anthropic today
#616how could they do it (not kill everyone) 1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear. 2) it hacks into public infrastructure taking down traffic, power, water, air traffic control, communications…
> 1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear. It does all of this using what compute? Frontier models require an insane amount of power and hardware to run - you can’t hack in to a TV and run My…
But you can silently sneak into devs accounts, steal tokens/keys and run small agents on their budget in some stolen VMs. Small so it's not noticeable.
Basically a virus spreading agents of some operation.
It should be in scope of imagination with anyone with brief knowledge how bot nets are made and behave.
Re: I resigned from Anthropic today
#617No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.
I love how there is always selective outrage here depending on if it's the favorite darling in question or not. // Anthropic / Apple - Nah, they can do no wrong. Even after both were caught violating users' trust and privacy multiple times, GOOD if (company in ["Anthropic", "Apple"]) do GOOD // Google / everyone else - Doesn't matter even if it's a good deed they've done, BAD else BAD end
Always?
Like 100 out of 100?
From every users?
Can you please consider this aspect too in your above pseudocode - no alternative execution path, or else - so there can be no misunderstanding about the always aspect? You know, some people use always to a majority part (>50%) of what they encounter, or even less when they weight that part dearly, but that does not account in the whole domain. Human chats may need clarification on trivial details like this.
Re: I resigned from Anthropic today
#618I know this is going to come off as jaded and dismissive, but when I saw the WSJ article my reaction was an eye-roll. The guy is 27 years old. He was poached by OpenAI, then poached by Anthorpic and now wants to retire early with the millions he's made.
Nothing wrong with wanting to retire early, but the pretence of suddenly caring about humanity at this stage seems attention-grabbing just for the sake of lining up some interviews (ie - more dollars).
Re: I resigned from Anthropic today
#619No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.
Eh, but as an overpaid engineered stuck in the SV bubble how would you get to see that?
If everything around you confirms your psychosis, and nobody gives you a reality check, how are you supposed to snap out of it?
Re: I resigned from Anthropic today
#620Earlier quoted context omitted.
Generally I don’t think anyone is arguing about the for now part. I don’t think it’s crazy to extrapolate out a few years and ask what kind of danger we’ll be in then. A team of 10,000 agents just solved the Navier Stokes problem (sans bad behavior by the researchers). Even 1 year ago that would have been unimaginable. What happens to this risk view as: 1. Robotics begin rolling out more broadly across the world. 2.…
The mathematics research results are certainly impressive, but I don't see what that has to do with robotics. Waymo is getting somewhere, but it's been a long slog. There doesn't seem to be much progress on, say, package delivery. For more, see: https://secondthoughts.ai/p/14-reasons-robotics-is-hard
Didn't you follow the robotics advances in the Ukrainian - Russian battlefield? There is now a death zone of 50km, only controlled by drones and automatic weapons.