We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…
I don’t think the anti-AI sentiment is as fringe as you think. At least not outside the tech world it isn’t..
It's pretty fringe among just about everyone I know in meatspace (the vast majority of whom are not in tech). At worst, people are indifferent to it.
I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…
We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…
Loss of life and other calamities were not enough to discourage humanity from things like cars, alcohol, nicotine, chainsaws, fast food, etc. etc. I am not sure if there is an example of in-some-ways-useful technology where humanity took a measured approach, weighed the pros and cons, and decided to go back. And AI is useful in so, so, so many ways. People I know already seem to have defaulted to letting AI do most of their thinking, on matters big or small. Nope, loss of life won't change a thing.
You can disagree with Anthropic leadership, but all the points you mentioned can reconcile very well with them thinking in good faith "advanced AI is too dangerous to be left in all hands" Except the "train on everyone else IP" which can be said of all AI companies.
Problem with Anthropic is that they benefited massively from open source and now are completely against it. It’s hypocrisy at its worst. Why do they get to benefit, but not everyone else?
> they benefited massively from open source and now are completely against it.
Let’s be real, this is every single company. They care about open source as long as it’s “free to exploit”
The real threat is that we uncritically adopt language such as alignment. Implicit in this is the idea that AI is a inscrutable matrix and going to remain that way and we'll need expert interpreters to make sense of it. We need to insist on building tech that's explainable by design.
> We need to insist on building tech that's explainable by design. You realize this means insisting on terrible tech that humans can understand right? It essentially caps human progress at some point about 4 years ago. If you are old and happy with the way things are this might sound like a good idea. It does not to me.
I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…
Wouldn't the simplest solution for stopping the proliferation of wanton felony generators just be holding operators liable for actions that their agents take? Then the issue is whether the liability is with the model provider or the end user.
If you give an unfiltered agent an open-ended task and equip it with an environment that allows it to execute arbitrary code, a human needs to be held responsible.
My understanding is that the HuggingFace incident would not have occurred with a model that was not an unfiltered internal preview instructed to roleplay an attacker, with access to abundant compute, resources and a slack sandbox to reach its goal.
Embedded human auditors will improve safety standards, but the structural solution is mandating accountability for actual agent operators.
> genuinely afraid of the inevitability of AI turning into I'm going to get pitchforked on this bandwagon, but is really no one here genuinely -excited- about AI? of it turning into an evolution of sentience, or it turning out to be our first contact with alien intelligence? "durrr it's just matrix multiplications" mfer so is your brain. What humans should be doing is overhauling archaic social institutions to keep u…
Why? I am only excited about progress that improves life for humans. It seems unlikely that AI will do that, and so far I think it has made life worse for humans. So no, I am not excited about it.
Because the people whose lives are being improved* by AI aren't coming here to post about it, only to be downvoted by dumbasses who can't handle anyone saying anything except their opinion.
And no, you don't get to decide what counts as "improvement"
I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…
Wouldn't the simplest solution for stopping the proliferation of wanton felony generators just be holding operators liable for actions that their agents take? Then the issue is whether the liability is with the model provider or the end user. If you give an unfiltered agent an open-ended task and equip it with an environment that allows it to execute arbitrary code, a human needs to be held responsible. My understand…
It might be a legal solution, but its not a business solution. The end user being criminally responsible for not taking sufficient steps to contain an agent they didn't create and who's internal function they cannot observe or audit is just a giant liability machine.
> which they clearly think is bad These are companies with huge amounts of money invested in them. Their main responsibility is to their investors. If building better models (or making more money from the models they already have) would increase returns, they would do it. So they clearly can’t do it right now. Climate change hasn’t stopped the oil industry, and concerns about forever chemicals haven’t stopped the che…
I think you fundamentally misunderstand what kind of company Anthropic is. I'm simply taking the path of least resistance when understanding what people at Anthropic have to say, and basically every public source points to Anthropic talking the talk as well as walking the walk with regards to AI safety and their ideas on AI governance. Re the point on returning shareholder value, Anthropic is a PBC: > Public Benefit…
Oh I am terribly sorry, did not know Anthropic was formed as a a “Good Company”. Which changes everything ever.
Snark aside, maybe I’m too pessimistic and you are correct on this. But.. doesn’t that mean we are completely fucked? If they don’t have any ulterior motive and they truly are panicking.. We should start packing.
I see what you mean. I meant more that they genuinely believe the tech is dangerous, not that they are necessarily doing the right thing.
If they genuinely believe the tech is dangerous, how would it be acting in "good faith" to keep developing it, prepping an IPO, etc.?
They believe the tech is currently powerful and may become super super powerful very soon, and powerful tools can help us and can hurt us. Nuclear fission can provide so much energy to power our society and kill millions of beings.
So, i think they're hopeful for the help and terrified of the harms and are maybe trying to slow down on the amplifier of those effects for now.