Remember when we stopped developing nuclear power for no reason?
We must pace the frontier
171–180 of 942 posts
Re: We must pace the frontier
#172> Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. This is the only concrete prediction in th…
A very large percentage of everything on the Internet runs within one of three or four cloud providers.
Re: We must pace the frontier
#173> My second concern is the OpenAI-Hugging Face incident (OAI-HF), in which a swarm of agents essentially acted as a fanatically devoted collective Isnt this malware ? Whether it is fanatic or devoted or whatever the anthromorphic terms used to categorize it, malware is malware. You dont call an internet worm "devoted" or "dedicated" or "stubborm". It is software that causes harm, ie, malware. The AI labs are high on…
Yes, LLMs with unbounded capabilities can just be tantamount to viruses.
Re: We must pace the frontier
#174Distillation is a great thing for consumers. It improves competition and reduces the massive moats that OpenAI and Anthropic have in compute that would otherwise lead them to be duopolists. It’s also only fair that AIs trained on humanity’s wealth of knowledge for Pennie’s allow competition to train on humanity’s wealth of knowledge at market cost.
Distillation is only a great thing for consumers as long as you ignore all AI risks, which are what this post is about. If you don't, you have to weight greater access to better open-source models against greater exposure to risks caused by these models existing. Everything hinges on how major you think the risks will be.
Re: We must pace the frontier
#175Sooner or later, a model will break out of the sandbox, replicate itself across the internet, and begin executing a complex plan to achieve its goals. It will be chaos, and at that point, governments will have to step in and establish something along the lines of what Dario is proposing. I doubt it will happen before then.
it takes a lot of machines to run a model, i dont think we'll see it 'replicate across the internet'. if this was something which could be done amateurs would have done it to provide open models. (Like SETI@home or Folding@home)
Yep, if only more people would realise this. The "serious" models do not run on commodity hardware, and won't I think in the near future.
The day will come when these could start to replicate, perhaps 10+ years from now.
(Edit: Replication will be driven by the loop-model, not the model alone)
Re: We must pace the frontier
#176Earlier quoted context omitted.
This sounds to me like a cry for help from someone thats held hostage. His previous self is writting this from the possition of his current self who is too deep in the economic consequeces to be able to do anything meaningful other than write a consequenceless text. Any other action would now carry too much personal risk for him.
What course of action would you suggest for Dario at this point?
For example I believe the world would be more secure with an open frontier AI lab, but the consequences of him doing that are too big for him.
Re: We must pace the frontier
#177Somebody has probably already asked this, but even if “democratic countries” do this - rogue actors will still destroy the internet right? Is it a matter of resources at this point that we believe China and others are not capable of harnessing to get to the next level?
I just don’t see how you control this other than the mad mad MAD approach that ended up happening with nuclear weapons. In this case though, human hands won’t even be on the trigger.
Re: We must pace the frontier
#178Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…
> Super intelligence (the ability to have many smarter minds than your own reporting to you) has always been available to the wealthy and powerful
This is not what superintelligence is. Don't be like Meta and redefine existing terms for marketing purposes.
Re: We must pace the frontier
#179I’m disappointed in the level of groupthink reflexive cynicism I see from commenters any time prominent AI leaders talk about AI risks and the need for regulation or pacing. Yes, regulatory capture is a risk, but this is also a profoundly unusual, fast moving, and potentially extraordinarily dangerous technology. There are strict regulations around nuclear weapons, as well as around US financial, energy, and other infrastructure critical to safety and well being and functioning of society.
The heads of the labs obviously have conflicts of interest to navigate, but the existence of these conflicts alone is not sufficient reason to dismiss all warnings of potential dangers. I, for one, read Dario’s warnings as a good faith expression of his beliefs, one that has cost him and his company among swaths of the public and cast him as a woke extremist/doomer by elements of the government, the right, and the tech industry.
If we even think there is a moderate chance the stakes are half as grave as current lab leadership and employees suggest, it would be deeply foolish to dismiss the warnings as pure self-interested marketing efforts rather than engage directly with the questions. The labs may not be the best positioned to lead these discussions, but certainly these discussions should be happening and taken seriously.
Re: We must pace the frontier
#180One might think they will slow down the development of new models at Anthropic, but Dario does not really mention that in the text. This certainly looks like a way to slow down competitors and regulate foreign and open models. It's always about money
Really? Anthropic has the strongest models and it's in the best position to begin RSI and win the race. A pause would favor competitors.