Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

131–140 of 943 posts

Re: We must pace the frontier

#131

Post by academia_hack »

angusturner wrote: post #115
academia_hack wrote: post #90

Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…

This seems highly optimistic to me. What's to say current AI won't be highly power-concentrating by default? The best models are owned by a few companies, and displacement of knowledge-workers mainly seems to benefit the capital class. On-device / edge computing makes sense in a few very limited scenarios. And economically, price or watt/token (or watt/task completed) might always be better in large data centers. No…

I'm not terribly optimistic. I don't think humans have really figured out what to do about historical materialism. More just remarking that this is a blog post that almost literally reads "fellow elites, let us ensure only we control the means of production. It is our duty to humanity to control the masses."

Re: We must pace the frontier

#132

Post by xg15 »

I'd like to have some info what those independent evaluators are supposed to do exactly - or which risks Dario specifically sees as being "unaligned". Evidently, with "pacing" he doesn't mean stopping the production of ever-more powerful models, so what exactly does he want to pace here?

Re: We must pace the frontier

#133

Post by ThrowawayR2 »

NiloCK wrote: post #45
blfr wrote: post #8

I enjoy Claude Code very much, have max privately and team premium at work, but the doomer marketing and this whole regulate-while-we're-ahead spiel is extremely annoying and makes me wish Anthropic gets trounced.

The degradation, polarization, and weaponization of or media landscape over the era of social media has left us utterly incapable of believing anything that anybody says. Dario in particular has consistently been risk-wary on model improvements for going on a decade - long before he was CEO of Anthropic. He believes what he is saying. Human beings can plainly say things that they think are true. Not every utterance b…

> "Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy."

Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation.

If he believes what he's saying, then when Anthropic is sued for their AI harming another party, his statements are evidence that Anthropic knew _in advance_ that their AI safeguards were likely insufficient to keep their product from harming people. It would be a blatant admission that they were reckless and negligent. That other AI companies are doing the same would not mitigate that.

Re: We must pace the frontier

#134

Post by youoy »

pr337h4m wrote: post #16

> Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. This is the only concrete prediction in th…

I like to replace thes AI text with "virus manipulation"

"Given the acceleratung rate of virus manipulation in labs, its my worry that in 6-12 months a virus could scape a take over the world and collapse health systems."

If a CEO of a health company was saying this, the reactions would not be that chill.

The worst failure of our society is to call this technology AI, intead of something line "artificial general automation". The formes allows the creators to be somehow less responsible of the consequences. The later clearly moves the responsibility to the creator/user.

Re: We must pace the frontier

#135

Post by [deleted] »

ArcHound wrote: post #129
xg15 wrote: post #118

I don't see how the dual goals of "we have to make an agreement with China for mutual slowdown" but also "we have to ensure we will always stay ahead of China" would work. I imagine the first thing the chinese government would demand in negotiations about such an agreement would be a lifting of the chip and distillation ban. > Some may believe these measures make it more difficult to cooperate with China, but I belie…

Some people would like to have law that protects them but doesn't bind and for others to be bound but not protected. They would like to slow down China while speeding ahead. Why wouldn't they want that? Now they have to make that happen somehow.

[deleted]

Re: We must pace the frontier

#136

Post by Avicebron »

bendergarcia wrote: post #76
vkaku wrote: post #47

Don't buy this argument. It's like saying, we lords who hold this capital will build all this economically destructive stuff anyway, and wait for the world to not react to our stupid ways of enriching yourselves. People are not going to slow down because this was brought to them on less than endearing terms. They don't see any of these stated noble intentions. Claude was already used to cause economic, political and…

I also don’t buy the arguments. The arguments about helping humanity cure diseases as if those are some of the leading Causes of suffering. The bar is so low that we don’t event need to cure diseases, we just need to lift the entire floor with basic solutions like: housing for everyone, continued education on healthy eating and just general continued education on like living. Life coaches so many very basic things th…

Yeah, it's insane people are using the argument that we'll pour trillions on moonshot cancer research when pouring trillions into universal healthcare would probably save countless more lives.

It's this kind of argument where the mask sort of slips because it always has to be an argument that also has the benefit of enriching himself and his friends.

Re: We must pace the frontier

#138

Post by glub »

GeneralMayhem wrote: post #102
epsteingpt wrote: post #82

The bioweapons threat is real, but shouldn't be constrained at the 'intelligence' level. Rather it should be constrained at the supply chain level. The cybersecurity threat will likely be a cat and mouse react game for a while. Just like robberies / the mob was in the early 20th century. Social forces bring things into balance over time, much more so than the proactive actions of individuals. Unfortunately, the genie…

What supply chain controls can you have on biotech? Part of the problem is that you can do a lot of damage with ingredients you can buy over the counter or make at home.

And all of this information has been available on the open web way before LLMs.

I'd wager it's likely easier for an average person to do this with Tor browser than it is to get an LLM to help them with it. Even ones that Dario calls dangerous.

Re: We must pace the frontier

#139

Post by kalkin »

RandomLensman wrote: post #92
kalkin wrote: post #79

Earlier quoted context omitted.

I'm curious to hear more about the differences in how bioweapons are regulated vs nuclear weapons.

For starters, they are banned by the bioweapons convention from the 1970s (180+ parties). Edit: I think back then the rather unpredictable nature and the little added value in deterrence etc. led people to the conclusions that arsenals of those things made little sense (and there was/is public dislike, too). The use was already banned by conventions from the earlie 20th century and the convention then addressed produ…

Ah yeah. I think the economic value of AI, unlike bioweapons, is probably too high for anything like that to be viable, but I think it's a reasonable thing to aim for. (Dario probably doesn't because he's a believer in short term positive biomedical impacts of AI in a way that I'm doubtful about.)

Re: We must pace the frontier

#140

Post by [deleted] »

pr337h4m wrote: post #16

> Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. This is the only concrete prediction in th…

[deleted]
Post reply on HN