Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

331–340 of 941 posts

Re: We must pace the frontier

#331

Post by Ydarbleoj »

stratos123 wrote: post #252
Ydarbleoj wrote: post #199

Earlier quoted context omitted.

He talks too much and as a result it's looked like pre-IPO hype and disingenuous because if he believed what he says then he'd stop. Now, someone is going to say but the investors and obligation to be first; and I would say exactly. The cynicism is well deserved and I think people are tired of what could be suggested is your group think take often called the status quo.

> disingenuous because if he believed what he says then he'd stop. Why do you think so?

Because I find the cognitive dissonance of saying it's dangerous while flying as fast as possible to an IPO too much to take. I just don't understand how others parse this differently.

Re: We must pace the frontier

#332

Post by freebsd_lovefes »

Dario is so out of touch with reality, living in some lalaland. Most people abroad will never truly and voluntarily comply with any sort of guardrails or pacing, no matter what case he or some leaders make. It's just not going to happen, maybe not even for show.

I do not want to make a case for the military to force that order on everyone, but if we are talking human extinction, which we are judging by what we see in mass media recently, then the military will take over.

Re: We must pace the frontier

#333

Post by simianwords »

echelon wrote: post #319
nicce wrote: post #314

Earlier quoted context omitted.

Many claims but no clear evidence that they actually find significantly more severe issues compared to open models.

Open models and agents can't be trusted without handholding. Astra can one-shot six months of work. Years of work, even. OpenAI just solved Navier-Stokes. Seems like the US is on a takeoff ramp to me.

Here are the cope points

1. Navier Stokes was plagiarism

2. All benchmarks were misleading wrong and incorrect

3. All other mathematical advances were again hype

4. HF incident was marketting ploy jointly coordinated by HF, METR and OpenAI (and also Anthropic)

5. Anthropic's HF like incident was again a marketing ploy [1]

Nothing ever happens. This whole thing is a scam. Everything is done to fool you and you have fallen for it. Congrats.

[1] https://www.anthropic.com/research/investigating-incidents-c...

Re: We must pace the frontier

#334

Post by abletonlive »

freebsd_lovefes wrote: post #332

Dario is so out of touch with reality, living in some lalaland. Most people abroad will never truly and voluntarily comply with any sort of guardrails or pacing, no matter what case he or some leaders make. It's just not going to happen, maybe not even for show. I do not want to make a case for the military to force that order on everyone, but if we are talking human extinction, which we are judging by what we see in…

What do you mean, Europeans would love to voluntarily comply with this because they have been of little relevance until American AI companies started to beg for regulation.

I am sure the Europeans rolling up their sleeves right now and getting to "work"

Re: We must pace the frontier

#335

Post by youoy »

shepherdjerred wrote: post #271
youoy wrote: post #265

Earlier quoted context omitted.

You mean China is not developing biological weaponds?

I’m not sure where you got that from or what your point is

Your point was that its an unfair comparison because AI its a race. My point is that bioweaponds are also a race, but a less public one. You dont have the equivalent of Dario publishing an essay every month.

Re: We must pace the frontier

#336

Post by cubic_earth »

stratos123 wrote: post #190
civiloai wrote: post #164

Earlier quoted context omitted.

it takes a lot of machines to run a model, i dont think we'll see it 'replicate across the internet'. if this was something which could be done amateurs would have done it to provide open models. (Like SETI@home or Folding@home)

It's not possible for models to replicate across consumer computers (without a major advance in distributed computing, at least), but that doesn't mean they can't replicate at all. There are services that'll rent you GPU pods by the hour with zero oversight, so even today, if a model can get access to some money and exfiltrate its weights, it can rent a bunch of GPU pods and run itself there. (It's not going to be tr…

There are very lightweight models out there. Not everyone in an army is a general. The swarm could be 100,000s of thousands of tiny models, and they could manage conventional botnet computers, and they could all take direction and guidance from a handful of frontier generals

Re: We must pace the frontier

#337

Post by avaer »

This is a "startups = wealth inequality" [1] formatted argument but:

This is 100% about money. The only way to pace the frontier is to make everyone (read: investors) lose all their money (read: no longer expect returns). Then nobody will pay the GPU bill or pay celebrities 10 million dollar salaries to stay at the hot lab. Suddenly the development is paced, almost like magic.

Conversely, it is hard to see how pacing development makes the current bubble justified, i.e. how development could be significantly paced without investors losing their faith in hot returns at current valuations. Faith in a bubble literally equals money, exactly in the way that loans created by a bank literally equals money. You can't have one without the other.

In fact if the whole industry goes bankrupt and investors are burned bigtime (many trillions wiped), this would spread transnationally, fixing the "if we don't do it, China will" loophole.

OpenAI had it right originally, the idea to be a NON profit and vow to never participate in an arms race. It's too bad that was tossed out the window now that there's money.

[1] https://paulgraham.com/ineq.html

Re: We must pace the frontier

#338

Post by rickydroll »

Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

Whenever someone talks about regulation, I always have the same question: How do you enforce it? How do you keep companies from using the replaced talent as a seat warmer? Hire 1 or 2 accountants, give them a chair and a cubicle, and let AI do the rest of the work.

Legislating against using AI to replace employees could create an unevenly distributed basic income. The idea of limiting AI to socially beneficial efforts is appealing, but I keep coming back to Bell Labs and Xerox PARC. Cool stuff was developed when you gave smart people a playground. Maybe the question should be: How do we give AI a playground and see what it comes up with? Although that could set us up with a situation like in the story Dragon's Egg.

Re: We must pace the frontier

#339

Post by ActionHank »

esseph wrote: post #207
ActionHank wrote: post #66

Earlier quoted context omitted.

Dude probably sleeps on a bed worth more than your networth. You have no frame of context to understand his intent or legitimate worries. The only applicable perspectives are to trust or apply logic. It is foolish to trust someone you don’t know who stands to benefit from lying to you. Logic dictates that given the ungodly sum of money he stands to gain, he will lie to everyone who will listen.

> Dude probably sleeps on a bed worth more than your networth. I never understand shit like this coming out of people's mouths. Never. It's not a judge of actual Worth as a human being, it's not a judge of capability or competence or ethics. It's not a judge of actual skill or ability. It doesn't make them a better cook, a better spouse, a better parent or lover. It doesn't make them more dangerous or more skilled at…

I think you’re misreading. No worship here.

Just pointing out that it’s foolish to think anyone in the position is even remotely thinking about anyone but themselves.

Re: We must pace the frontier

#340

Post by Paradigma11 »

nickysielicki wrote: post #157
angusturner wrote: post #115

Earlier quoted context omitted.

This seems highly optimistic to me. What's to say current AI won't be highly power-concentrating by default? The best models are owned by a few companies, and displacement of knowledge-workers mainly seems to benefit the capital class. On-device / edge computing makes sense in a few very limited scenarios. And economically, price or watt/token (or watt/task completed) might always be better in large data centers. No…

> The best models are owned by a few companies In terms of cost per task, the open weight Chinese models are winning by a long shot. So it depends on what you mean by the best model.

Are you sure about that?

when I checked a few weeks ago I did not really find anything competing per value with a 20x Codex subscription.

Post reply on HN