Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

211–220 of 935 posts

Re: We must pace the frontier

#211

Post by 0xDEAFBEAD »

Matl wrote: post #162
0xDEAFBEAD wrote: post #93

Earlier quoted context omitted.

How specifically is Dario a hypocrite? 129857's case rests on Dario being an "idealist". But maybe he's an idealist about curing cancer ASAP, and not an idealist about respecting copyright. That's not necessarily hypocritical.

For example he got into a spat with the Department of War as if he cared for how his AI could be used during war and yet said he's fine with Claude targeting a girl's school in Iran. That's apart from the general fact that he continues to race towards the very thing he claims he's afraid of, because that's where his net worth comes from.

>said he's fine with Claude targeting a girl's school in Iran

Where?

Re: We must pace the frontier

#212

Post by sailfast »

youoy wrote: post #134
pr337h4m wrote: post #16

> Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. This is the only concrete prediction in th…

I like to replace thes AI text with "virus manipulation" "Given the acceleratung rate of virus manipulation in labs, its my worry that in 6-12 months a virus could scape a take over the world and collapse health systems." If a CEO of a health company was saying this, the reactions would not be that chill. The worst failure of our society is to call this technology AI, intead of something line "artificial general auto…

Can we build level IV AI containment labs?

Re: We must pace the frontier

#213

Post by shepherdjerred »

glub wrote: post #138
GeneralMayhem wrote: post #102

Earlier quoted context omitted.

What supply chain controls can you have on biotech? Part of the problem is that you can do a lot of damage with ingredients you can buy over the counter or make at home.

And all of this information has been available on the open web way before LLMs. I'd wager it's likely easier for an average person to do this with Tor browser than it is to get an LLM to help them with it. Even ones that Dario calls dangerous.

You are right that it’s easier to do this via Tor than an LLM. Those are the safeguards…

Re: We must pace the frontier

#214

Post by kart23 »

> Do not sell powerful AI chips or semiconductor manufacturing equipment to China, and crack down on chip smuggling operations and remote access to data centers outside China. Chips will be the main determinant of China’s AI strength.

> If we execute these measures well, I believe they would slow China’s progress enough to widen America’s lead significantly over the next 3–5 years — the window when AI becomes geopolitically most important.

china is literally making their own ASICs now, not sure that this is the silver bullet he proposes.

https://www.silicon.co.uk/ai-2/huawei-cambricon-ai-630499/am...

Re: We must pace the frontier

#215

Post by shepherdjerred »

api wrote: post #198
oceanplexian wrote: post #189

Earlier quoted context omitted.

The bioweapons argument is one of the favorites used by these con artists and dreamed up by their PR team. Except Bioweapons already existed before LLMs, Adversarial governments already have them, they are already easy to make. You could use the same bullshit argument for why we need to ban libraries, books, or require a license to buy an internet connection.

An undergrad bio student with some lab experience can do some effing terrifying things with about $20k in equipment and time and access to some papers and a library. AI is not needed, but it might help accelerate the research. Not going to go into it but I studied biology. It’s all out there. It’s easier than you think. It hasn’t happened yet because… nobody has done it. That’s the answer. There is no policeable phys…

That’s not entirely true though, right? There have been plenty of bioweapon attacks

Ex https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack

Re: We must pace the frontier

#216

Post by ks2048 »

andxor wrote: post #180
basedpolymer wrote: post #5

One might think they will slow down the development of new models at Anthropic, but Dario does not really mention that in the text. This certainly looks like a way to slow down competitors and regulate foreign and open models. It's always about money

Really? Anthropic has the strongest models and it's in the best position to begin RSI and win the race. A pause would favor competitors.

> win the race

There is no finish line. Anthropic gets somewhere and others get "there" (or somewhere near "there") a little bit later.

Re: We must pace the frontier

#217

Post by credit_guy »

Jcampuzano2 wrote: post #63
TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

If someone is genuinely afraid of this, they wouldn't IPO in the first place. All the talk in the article about commercial incentives means nothing when the company plans to IPO and become beholden to investors.

I don't think Anthropic and OpenAI will IPO at all. Word is that Anthropic will have $100 BN in revenues this year, and very likely OpenAI will get some similar amount. You IPO when you need money, and I think Anthropic and OpenAI are past that point.

Re: We must pace the frontier

#218

Post by itajaja »

ThrowawayR2 wrote: post #133
NiloCK wrote: post #45

Earlier quoted context omitted.

The degradation, polarization, and weaponization of or media landscape over the era of social media has left us utterly incapable of believing anything that anybody says. Dario in particular has consistently been risk-wary on model improvements for going on a decade - long before he was CEO of Anthropic. He believes what he is saying. Human beings can plainly say things that they think are true. Not every utterance b…

> " Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy. " Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. If he believes what he's saying, then wh…

> it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation.

not really. if you really want to go by definition in the book and apply it the CEO role, Amodei is CEO of an LTBT, so it's in the CEO's goal to benefit humanity. You can believe in his sincerity or not if you want, but using the CEO label as a definitional reason as to why he must lie is factually incorrect.

Re: We must pace the frontier

#219

Post by stratos123 »

Sevii wrote: post #13
heaney-555 wrote: post #11

None of this works without buy-in from China. This isn't something private companies can decide. The US would need to sign a groundbreaking deal with China, equivalent to the Anti-Ballistic Missile Treaty of the Cold War.

We'd have to make a deal with China and be confident they wouldn't cheat on it.

That's not undecidable in principle - compute governance is a thing. The more likely sticking point is that the two sides might be soured on the deal once they realize how much oversight they'd have to give to the other side.

Re: We must pace the frontier

#220

Post by oceanplexian »

TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

It's a chatbot that escaped a misconfigured Docker container.

I run open source models and it's pretty obvious when they fail some tool call and then keep trying different permutations of things until they get a solution. Just like Metasploit if you try enough different things at scale you will eventually crack some software or find a vulnerability in something. If you spend billions of dollars on compute and run tens of thousands of agents you might solve some novel Math and STEM problems as well, big whoop.

What Anthropic and OpenAI really need is to hire some engineers that know what they're doing and know how to properly set up a proxy server and sandbox/microVM, not a Global Arms Treaty.

Post reply on HN