Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

361–370 of 942 posts

Re: We must pace the frontier

#361

Post by glub »

echelon wrote: post #319
nicce wrote: post #314

Earlier quoted context omitted.

Many claims but no clear evidence that they actually find significantly more severe issues compared to open models.

Open models and agents can't be trusted without handholding. Astra can one-shot six months of work. Years of work, even. OpenAI just solved Navier-Stokes. Seems like the US is on a takeoff ramp to me.

Tell me you haven't tried letting Astra go without telling me.

Astra can confidently one-shot 500k lines of slop, with 800k lines of tests covering it, without testing a single intended product requirement, and none of it actually working.

All models require hand holding. Fable and Astra are no exceptions. The difference is only in the amount of hand holding required, and there's essentially no gap here anymore between American and Chinese models.

I only use Chinese models sparingly because American models are so much cheaper with subscriptions, that it doesn't make economic sense to not use them. If/when that changes, I could simply route to cheapest model that's available at the moment and I wouldn't notice much difference in most applications.

Re: We must pace the frontier

#362

Post by bluerendition »

academia_hack wrote: post #90

Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…

[dead]

Re: We must pace the frontier

#363

Post by verdverm »

pastel8739 wrote: post #258
iloveoof wrote: post #75

Distillation is a great thing for consumers. It improves competition and reduces the massive moats that OpenAI and Anthropic have in compute that would otherwise lead them to be duopolists. It’s also only fair that AIs trained on humanity’s wealth of knowledge for Pennie’s allow competition to train on humanity’s wealth of knowledge at market cost.

He did actually specify that diffusion should be limited for authoritarian countries, which I appreciated. If his focus is really safety, distillation in countries that are bound by safety regulation should be fine

> bound by safety regulation should be fine

I do not believe this is a feature we can differentiate between the "good" and "bad" guys, as the US is currently under a poor safety regulation regime. Democracies can elect unethical people, write bad laws, and have uncertain enforcement. In example, the current US admin regularly lambasts Europe because they try to have stronger regulation.

Re: We must pace the frontier

#364

Post by logicchains »

65 wrote: post #358
academia_hack wrote: post #90

Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…

This feels a bit hyperbolic to me. AI seems to me to be similar in "revolutionary" terms as the web, which made information widely available for free if you had an internet connection.

There's a big difference from just having access to information, and also having access to the kind of intelligence that would previously have cost hundreds of dollars per hour to hire.

Re: We must pace the frontier

#365

Post by cubic_earth »

This is either performative or shallow.

What does "aligned" even mean? Aligned with who? We have no universal code of ethics. Democracies kill and launch wars merely to have cheaper stuff, even when we are already rich. And why would China ever agree to stay in second place? Our society is built on the premise "the smart and powerful dominate". The idea of domination is deeply embedded in our capitalist model.

Or course it is true that powerful AI will empower the average Joe to make a bio-weapon, and that will lead to our ruin. But at the same time, having powerful AI in the hands of a just a few is almost equally as horrific.

But deeply embedded in out national and cultural ethos is to advance science, tech, and material wealth at all costs. We never ask if we have enough. We sacrifice community to advance our careers for ever more.

There is no way out of this one, I'm afraid. Our cultural predispositions and mindset of domination compel us to chase ever more powerful AI as if doing so were a mandate from god.

And the result will be a crisis in so many dimensions it is hard to reason about what will go wrong first and most spectacularly.

Re: We must pace the frontier

#366

Post by csto12 »

braydenm wrote: post #270

Before Anthropic, I worked at Cruise for four years as it competed against Waymo. The culture rewarded (and demanded) moving quickly, trusting that the company could empirically discover the risks that the robot cars posed and iteratively solve them to keep up with the rate at which it was scaling out its technology. There was very little interest or appetite for coordinating or collaborating with other AV companies…

It’s not explicitly stated, but if you are insinuating that you don’t know how many more Saturdays you will have left to play outside because of the AI race while working for Anthropic, do you not have agency to quit? How could you be apart of something that makes you feel that way? That blows my mind.

Re: We must pace the frontier

#367

Post by vatsachak »

csto12 wrote: post #366
braydenm wrote: post #270

Before Anthropic, I worked at Cruise for four years as it competed against Waymo. The culture rewarded (and demanded) moving quickly, trusting that the company could empirically discover the risks that the robot cars posed and iteratively solve them to keep up with the rate at which it was scaling out its technology. There was very little interest or appetite for coordinating or collaborating with other AV companies…

It’s not explicitly stated, but if you are insinuating that you don’t know how many more Saturdays you will have left to play outside because of the AI race while working for Anthropic, do you not have agency to quit? How could you be apart of something that makes you feel that way? That blows my mind.

He's insinuating that AI will end the world lol. Nothing ever happens

Re: We must pace the frontier

#368

Post by TuxSH »

Paradigma11 wrote: post #340
nickysielicki wrote: post #157

Earlier quoted context omitted.

> The best models are owned by a few companies In terms of cost per task, the open weight Chinese models are winning by a long shot. So it depends on what you mean by the best model.

Are you sure about that? when I checked a few weeks ago I did not really find anything competing per value with a 20x Codex subscription.

In terms of API costs they're about right, 10% of weekly usage on a 5x sub is roughly $40 (for Astra). Chinese models cost roughly half as much (and have fewer refusals).

Subs have insane value because large companies cannot use the subscription model. They are loss leaders and are often used for passion projects & startups (where the juicy data is).

Ah and OAI stopped offering the 20x sub as of yesterday.

Re: We must pace the frontier

#370

Post by afdgnionio »

glub wrote: post #302

Dario, Sam, and Elon are all on the same page on this. So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky, so what could the other possible motives be, based on what we know? OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models…

[dead]
Post reply on HN