Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

91–100 of 942 posts

Re: We must pace the frontier

#91

Post by gewa »

Jcampuzano2 wrote: post #63
TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

If someone is genuinely afraid of this, they wouldn't IPO in the first place. All the talk in the article about commercial incentives means nothing when the company plans to IPO and become beholden to investors.

We are still living in a capitalistic society. We have to find a solution which is responsible, safe and returns on the investment. The commercial aspect can be true at the same time.

Re: We must pace the frontier

#92

Post by RandomLensman »

kalkin wrote: post #79
RandomLensman wrote: post #62

Why is the analogue necessarily the regulation and control of nuclear weapons (e.g., SALT) and not, for example, that of bioweapons? Some very different paths are available. On both I would note that the private sector has only a limited role, though.

I'm curious to hear more about the differences in how bioweapons are regulated vs nuclear weapons.

For starters, they are banned by the bioweapons convention from the 1970s (180+ parties).

Edit: I think back then the rather unpredictable nature and the little added value in deterrence etc. led people to the conclusions that arsenals of those things made little sense (and there was/is public dislike, too). The use was already banned by conventions from the earlie 20th century and the convention then addressed production, development and stockpiles (incl. delivery systems, I think).

I guess all I wanted to point out is, that there can actually be agreement to ban certain technological things pretty comprehensively (outside of some peaceful protective research etc.). Whereas things like SALT are (or were in that case) limiting the number of weapons deployed.

Re: We must pace the frontier

#93

Post by 0xDEAFBEAD »

largbae wrote: post #86
kalkin wrote: post #69

Earlier quoted context omitted.

If Anthropic shut down (or even counterfactually had never been founded), would software development be freed from the impact of AI?

Probably not, but we wouldn't have to listen to Dario's hypocrisy while it happened

How specifically is Dario a hypocrite? 129857's case rests on Dario being an "idealist". But maybe he's an idealist about curing cancer ASAP, and not an idealist about respecting copyright. That's not necessarily hypocritical.

Re: We must pace the frontier

#94

Post by kalkin »

pr337h4m wrote: post #50
kalkin wrote: post #37

Earlier quoted context omitted.

Why should we believe that a scaled out version of something that happened a few months ago "simply cannot happen"? How many dollars of compute do you believe were available to the swarm(s) behind the OAI-HF, German wiki, and Rubygems incidents?

Do you realize how big "the entire internet" is? > How many dollars of compute do you believe were available to the swarm(s) At least two OOMs more than the dollar value of the damage they'd caused. (Also, as an aside, IIRC, the wiki servers weren't breached; it was just a lot of spam.)

Sure. And there's an OOM more compute coming online in the next year or two, while models at a given capability are getting cheaper. "Two OOMs" of scale relative to the HF swarm seems like a bit of a red herring to me, but also within the realm of possibility.

The Internet is big, but one can do quite a lot of damage with ordinary bots and worms that exploit individual widespread vulnerabilities, which LLMs are perfectly capable of writing. Most of the damage also doesn't rely on hitting every long-tail website.

I'm honestly not that concerned about cyber impacts of LLMs relative to other impacts. I just don't like to see the whole concept of being worried dismissed as obviously baseless on the basis of one pretty shaky scale argument.

Re: We must pace the frontier

#95

Post by 0xDEAFBEAD »

PunchyHamster wrote: post #88
TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

I dunno man, it all just sounds like trying to put controls on AI while being the favourite child of govt so competition can't fight as easily. Especially with IPO around the corner

If Dario was primarily motivated by being the "favorite child of govt", he would've yielded during the DoD showdown.

Re: We must pace the frontier

#96

Post by amanfromsolan »

jraines wrote: post #35

Frankly, I am sick of "it's all just marketing and attempt to do regulatory capture, and this is obvious to me, a smart person" on every single discussion related to safety. There could be an element to truth to it, but it's certainly not the entire story and is just so tiresome at this point.

Couldn’t agree more. I feel my heart sinking where anytime anyone suggests any regulation for a potentially, suggested by multiple people, world changing or destroying tech, all the moments here are “ah you billionaire, ha! trickster

If you believed it, you would SHUT anthropic or release all models open source.”

And then what? How does that help with slowing down the frontier? Should he shut shop and then grovel Sam’s feet to make it work and become an activist?

Maybe he actually believes the world changing power of AI and hence shoved his entire time into it and half of it has worked out since Anthropic is going to be worth a trillion and he’s terrified of the other half being true too?

Re: We must pace the frontier

#98

Post by academia_hack »

kalkin wrote: post #65
academia_hack wrote: post #39

Earlier quoted context omitted.

The essay includes specific regulatory measures to prevent developing countries from accessing hardware and software needed to benefit from technological development. The US government is constantly applying new perversions of arms control regulations based on the preposterous claim that an LLM is somehow an "arms" in order to restrict Chinese access to GPUs. They are willing to prevent virtually any country on earth…

Export controls vs China don't imply Dario's personal control of AI, nor are they broadly in the interest of "tech billionaires" (Jensen hates them). Neither does admitting external auditors at AI companies--if anything that's a surrender of control. I don't see anything in the essay about restricting open frontier weights. If you want to argue that the US has no right to be restricting technological development else…

I think you're reading too much into what I'm saying. We've got a whole legal system developed over the course of thousands of years which is well suited towards regulating the effects of "intelligence" by regulating what people _do_ with it.

Lobbying for a completely parallel regulatory framework to restrict who is allowed to run specific types of general purpose computer programs, just because we can imagine that some of those "general purposes" might be bad, feels a lot more like an attempt by capitalists to carve out a special kingdom over this specific technology for themselves.

For example, I'd be 100% in favor of holding anthropic criminally liable any time their AI enables someone to commit a crime. But I'm opposed to banning the export of GPUs above a certain size just because Dario doesn't trust what someone like me would do with them if he couldn't snoop on my conversations.

Re: We must pace the frontier

#99

Post by kalkin »

PunchyHamster wrote: post #88
TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

I dunno man, it all just sounds like trying to put controls on AI while being the favourite child of govt so competition can't fight as easily. Especially with IPO around the corner

Do you think Anthropic's behavior in the last year is well explained by aiming to be "the favorite child of govt"?

Re: We must pace the frontier

#100

Post by 0xDEAFBEAD »

reticulates wrote: post #59
yewenjie wrote: post #51

Earlier quoted context omitted.

Because he genuinely believes if he doesn't do it the next guy will do it worse.

There’s an astounding level of arrogance required to believe he is somehow uniquely capable of bringing about a technology especially considering Anthropic came after OpenAI where he worked.

Beating OpenAI is not exactly a high bar here: https://www.openaifiles.org/

I don't think it is particularly egotistical to say that you can be a more ethical CEO than Sam Altman.

Post reply on HN