Earlier quoted context omitted.
I believe that the actual alignment happens in.. uh.. "meatspace". Someone is prompting. Someone is hosting. That someone needs to be accountable for what happens. That someone needs to bleed if stuff goes haywire. Humans at large have been "aligned" by the shared fear of death, pain and suffering. This has proven to work for millennia, so all we need to do is reapply it.
You're still thinking in terms of control. That's the wrong model here. How do you control someone infinitely smarter than you? How do you control ten trillion someones?
I resigned from Anthropic today
481–490 of 1001 posts
Re: I resigned from Anthropic today
#482You want to win an AI benchmark, but not sure if you're that good? You'd go after the codebase and artifacts that runs the benchmarks, thus the agents went straight to Artifactory, they needed public Internet access... They failed many times, but were able to persist their "collective" state, and apparently some of the subagents with cheaper models were literally prompted to do grunt work or die, for which you have to wonder what must be in those training instructions to make it effective. Remember that nothing I said so far ever points out to LLMs being intelligent, it's the harness that has a few tricks up his sleeve. LLMs don't need to be intelligent, the harness that runs it absolutely needs to make up for that.
But this guy? He's timed his exit, waiting for the IPO, that's for certain. He's probably even feeling good about himself, hedging between altruism, AI concern hamstering and guerilla marketing. If you're quoting science-fiction over this, I'm sorry to inform you that you have absolutely no idea what's going on here.
Re: I resigned from Anthropic today
#483Earlier quoted context omitted.
> there is evidence bioengineering is already happening Mind sharing this evidence with us?
Does https://news.ycombinator.com/item?id=30698803 count?
Also
> That is, I'm not sure that anyone needs to deploy a new compound in order to wreak havoc - they can save themselves a lot of trouble by just making Sarin or VX, God help us.
We already have toxic nerve agents that are largely available for state actors and possibly available for individuals. If you have decided, as a human, to make great harm, you can already do that.
Re: I resigned from Anthropic today
#484“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
Re: I resigned from Anthropic today
#485Earlier quoted context omitted.
You're still thinking in terms of control. That's the wrong model here. How do you control someone infinitely smarter than you? How do you control ten trillion someones?
I press ctrl + c and inference stops.
Re: I resigned from Anthropic today
#486Earlier quoted context omitted.
This is not correct. Please go through each one again, taking special note of global warming and nuclear weapon development .
Why do you think global warming could kill literally every human being? What's the scenario in which that happens?
Or, once billions of people die from climate change, planetary wars will start on the last hospitable areas, leading to the end of humanity.
In contrast, AI ending "humanity" is limited by our own material power, would likely be stoppable, and would not easily reach areas isolated from technology
Re: I resigned from Anthropic today
#487You can say "yes and you have to fear the humans weilding the AI" - that I agree with.
Re: I resigned from Anthropic today
#488Earlier quoted context omitted.
How is global warning gonna kill all of humanity? Even a full scale nuclear war wouldn't manage that. They could destroy most civilisations, culture and scientific achievements though.
In the exact same way major sudden changes in the climate have lead to the extinction of the majority of plants an animals over this planets history. We are not special.
Possible? Theoretically yes, but pretty unlikely. But obviously extreme global warming would be a catastrophe even if some of humanity survive.
Re: I resigned from Anthropic today
#489Earlier quoted context omitted.
To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.
Note that OpenAI has jettisoned every other supposed value they had (releasing their work as open source, not working on military applications, being a nonprofit). I'm sure we can rely on them this time.
Re: I resigned from Anthropic today
#490Is it so implausible to imagine the following scenario, in the not too distant future? 1) AI models get extremely good at cyber attacking every system and start communicating in just binary. 2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accompl…