[deleted]
We must pace the frontier
351–360 of 943 posts
Re: We must pace the frontier
#352One might think they will slow down the development of new models at Anthropic, but Dario does not really mention that in the text. This certainly looks like a way to slow down competitors and regulate foreign and open models. It's always about money
Really? Anthropic has the strongest models and it's in the best position to begin RSI and win the race. A pause would favor competitors.
Anthropic is far behind OpenAI now, that's why it's OpenAI that's solving Millennium Prize problems, and why Astra completely blows away Fable on benchmarks.
Re: We must pace the frontier
#353Dario is only interested in regulatory capture.
Re: We must pace the frontier
#354Earlier quoted context omitted.
> The best models are owned by a few companies In terms of cost per task, the open weight Chinese models are winning by a long shot. So it depends on what you mean by the best model.
Are you sure about that? when I checked a few weeks ago I did not really find anything competing per value with a 20x Codex subscription.
[dead]
Re: We must pace the frontier
#355Sooner or later, a model will break out of the sandbox, replicate itself across the internet, and begin executing a complex plan to achieve its goals. It will be chaos, and at that point, governments will have to step in and establish something along the lines of what Dario is proposing. I doubt it will happen before then.
it takes a lot of machines to run a model, i dont think we'll see it 'replicate across the internet'. if this was something which could be done amateurs would have done it to provide open models. (Like SETI@home or Folding@home)
> if this was something which could be done amateurs would have done it to provide open models. (Like SETI@home or Folding@home)
I know that's not what you meant but this does exist, by the way. It's called AI Horde: https://github.com/Haidra-Org/AI-Horde/tree/main
The big difference is that a particular query is handled by just one particular node (a single model doesn't get distributed among the network), so it can only serve models small enough to be handled by a single consumer PC.
Re: We must pace the frontier
#356Earlier quoted context omitted.
Why should we believe that a scaled out version of something that happened a few months ago "simply cannot happen"? How many dollars of compute do you believe were available to the swarm(s) behind the OAI-HF, German wiki, and Rubygems incidents?
Well a big reason that we criticize OpenAI for that is because they were the ones giving it access to the massive compute necessary for the LLMs to think. If they had been responsible about their experiments or what types of workloads they allow their LLMs to operate, it wouldn't have happened. Very few companies could enable those workloads.
Well sure, but access to that compute is gated by simple credentials (like API tokens). Those can be hacked.
Imagine for example if a model hacks into ~every Linux computer on the internet using an 0day and steals their OpenAI, anthropic and openrouter credentials. It now has access to billions of compute and the only way to fully stop that is for multiple major providers to shut down services entirely. That's already well into "billions of dollars of damage" territory.
Re: We must pace the frontier
#357Earlier quoted context omitted.
It's not possible for models to replicate across consumer computers (without a major advance in distributed computing, at least), but that doesn't mean they can't replicate at all. There are services that'll rent you GPU pods by the hour with zero oversight, so even today, if a model can get access to some money and exfiltrate its weights, it can rent a bunch of GPU pods and run itself there. (It's not going to be tr…
If the models start buying up a bunch of pods and causing damage, couldn't those services just shut off their access?
Sure they can, after the fact.
Re: We must pace the frontier
#358Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…
This feels a bit hyperbolic to me. AI seems to me to be similar in "revolutionary" terms as the web, which made information widely available for free if you had an internet connection.
Re: We must pace the frontier
#359Dario is so out of touch with reality, living in some lalaland. Most people abroad will never truly and voluntarily comply with any sort of guardrails or pacing, no matter what case he or some leaders make. It's just not going to happen, maybe not even for show. I do not want to make a case for the military to force that order on everyone, but if we are talking human extinction, which we are judging by what we see in…
What do you mean, Europeans would love to voluntarily comply with this because they have been of little relevance until American AI companies started to beg for regulation. I am sure the Europeans rolling up their sleeves right now and getting to "work"
:)
Europeans are a non-representative subset.
Re: We must pace the frontier
#360I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…
>Don’t use AI to replace every job, from graphic designer to accountant to software developer. Place legal restrictions on (at least) large companies such that they may not use AI to replace human labor in this way.
It's not just "companies". Even if the company does nothing, enterprising employees are going to use LLMs to magnify their productivity, which will may reduce the company's need to hire more employees. And except in extremely locked down environments, there's absolutely no way to stop an employee using some open source LLMs to multiply their productivity.