Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone should slow down AI development except for me

xeiaso.net

Post reply on HN
Print view View original

431–440 of 459 posts

Re: Everyone should slow down AI development except for me

#431

Post by jurgenburgen »

user43928 wrote: post #250
nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

Open-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.

Still waiting for our org to roll out Mythos. I guess it was too expensive so we’re stuck on the previous model until the internal team can figure out self-hosting open models.

Re: Everyone should slow down AI development except for me

#432

Post by lukan »

ipaddr wrote: post #421
BoiledCabbage wrote: post #213

Earlier quoted context omitted.

> mediocre - of moderate or low quality, value, ability, or performance https://www.merriam-webster.com/dictionary/mediocre Clearly they were using the word to mean low quality. Why would you ask this odd question?

Core meaning: Barely adequate, average, or just acceptable. Mediocre means of only ordinary or moderate quality—neither very good nor very bad, and often slightly disappointing The quality is average but expectations of high quality are not met. He expected more but got what he asked for. We overuse top models because of this.

The claim was there is no difference between Opus and Fable but to me there is.

Opus delivered mediocre results. Not garbage, but would have required me to do lot's of things myself. Fable did not needed my supervision with this task.

Re: Everyone should slow down AI development except for me

#433

Post by phoghed »

flir wrote: post #401
ben_w wrote: post #346

Earlier quoted context omitted.

They're trying to make it not be a threat, and are all scared and afraid that their best efforts to make it harmless are not enough. Some are worried by the AI directly bringing doom; others are worried that one of the companies who control the AI will become a dictator; still more think becoming a dictator is a necessary step to safely prevent anyone else making unsafe AI. Painting them all under one brush is like d…

Ok, so they're uniquely careful, and they also get to that threshold first (whatever it is). What happens when everyone else (who is not so careful) gets to the same threshold three months later? How does them getting there first stop that happening? It's an utterly self-serving argument and it's not even internally consistent.

Obviously you have to turn your WMD against them to prevent this, for the good of everyone, because you're such a good guy.

Re: Everyone should slow down AI development except for me

#434

Post by Turn_Trout »

nullbio wrote: post #320
Turn_Trout wrote: post #204

Earlier quoted context omitted.

Dario's post [1] commits to direct evaluators that can, among other abilities, expose secret RSI. He wants that made law. Do you have a source on METR employees retaining massive equity stakes? [1] https://darioamodei.com/post/we-must-pace-the-frontier

Joe Benton left Anthropic a day before Dario's post, to work for METR evaluations. He was with Anthropic for over a year. He did the same thing that Jacob did (big song and dance about AI apocalypse, media interviews all over the place). He managed the Scalable Oversight team at Anthropic and was the research lead for the Anthropic Fellows Program. So he has equity, and likely lots of it. Then you have Josh Engels qu…

> Good Ventures is a funding partner of Open Philanthropy, who funded Jacob Coxon (the first of the Anthropic employees going viral in the media) via a scholarship.

This conspiracy theory is truly crazy. A $20K scholarship in 2022 is supposed to explain Coxon walking away from unvested equity for a company worth over $950 billion dollars?

Re: Everyone should slow down AI development except for me

#435

Post by PoignardAzur »

nobodyandproud wrote: post #406
PoignardAzur wrote: post #393

The HN discourse on AI is so exhausting. It's all knee-jerk and headlines only. Just people rushing to push their own beliefs and pre-emptively insulting anybody who disagrees. People here are so desperate for the AI race to continue forever, for anything that pisses off the billionaires that got us there even if that means innocent people go down with them because the next AI decides to blackmail a hospital to cheat…

I think both can be true: AI development must be slowed, but also question the motives of those in the lead given the stark benefits it gives those in the lead. I'll get down-voted again by those who have a stake in this race, but I believe the long-term freedom and benefit for "everyone" will come from being able to run models locally.

I agree that we need more nuanced complex takes on the situation... But also, if the problem is that AIs are starting to hack random website against their owners' instructions, open-weights AI isn't going to help that much.

Re: Everyone should slow down AI development except for me

#436

Post by datadrivenangel »

nullbio wrote: post #69
AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

Open weight models are all built on IP theft as well. Also more like 6-12 months behind, maybe 18 but it's hard to say.

Re: Everyone should slow down AI development except for me

#437

Post by fc417fc802 »

metalcrow wrote: post #410
fc417fc802 wrote: post #224

Earlier quoted context omitted.

> frontier research is halted but efficiency isn't Why are you treating those as if they're separate things? > Advancing the frontier seems to take a massive amount of compute, data, and parameters: miniaturization only happens afterwards. This is just completely wrong. Don't mistake the path by which something happened (or appeared to an outsider to have happened) for a fundamental truth. The frontier labs build mas…

> Why are you treating those as if they're separate things? Because they are? You haven't explained at all your belief that they are the same. > This is just completely wrong. Don't mistake the path by which something happened (or appeared to an outsider to have happened) for a fundamental truth. So it's completely wrong but it's only what has happened so far? You're repeating yourself but you're not listening to my…

To both points, you appear to broadly lack technical understanding of the field. You are assuming that how things appear to you reflects how they are at a lower level.

A chronological review of the architectures of the best open models over the past 3 years should immediately make the problem clear to you. (If you really refuse to put in any effort then perhaps just dump this thread into a frontier model rather than responding.)

> I literally responded to your claim that "would you have us destroy all chip fabs and ban all biomedical research?"

I don't see it? It's a rhetorical question to drive home the point about the human brain. Chip fabs and biomedical research are both viable pathways to more efficient hardware. Thanks to biology we know what's possible. If you outlaw something but don't effectively cut off all the pathways to it then someone will still do it if it's profitable.

Re: Everyone should slow down AI development except for me

#438

Post by int_19h »

anon373839 wrote: post #381
int_19h wrote: post #280

Earlier quoted context omitted.

I don't think you can ever make a case for it on cost savings in general. Inference is very obviously the kind of problem where things are cheaper at scale, and this is still true for smaller models. The only reason to run locally is privacy.

Privacy is a great reason, but independence is another. It’s very nice knowing that you’re going to get the same reliable product every time you call the model. Nothing is going to change unless you decide to change it.

That is not a counterargument to cloud inference though. You can also run open weight models in the cloud, and it's still cheaper. So privacy really is the only motivation to run on local hardware.

Re: Everyone should slow down AI development except for me

#439

Post by Animats »

jandrese wrote: post #223
Waterluvian wrote: post #111

Earlier quoted context omitted.

I think space being imagined as some magical international waters that protects something from government powers is poisonous thinking. It just means it’s protected by no power instead of a power with an agenda. Practically, if it was ever possible to build such a thing, it would take a fraction of the effort to destroy it.

I mean for that matter why not float barges in the South Pacific for data centers? Sure it’s a dumb idea, but it’s still less problematic than launching thousands of GPUs on rockets.

Look up "seasteading". It's not a win.

Low earth orbit, though... Solar power works well, radio works well, and the international jurisdiction is confused enough to exploit if you have money.

Re: Everyone should slow down AI development except for me

#440

Post by int_19h »

the_optimist wrote: post #329
int_19h wrote: post #268

Earlier quoted context omitted.

Anthropic specifically is basically saying that they believe it's even more dangerous if someone else gets to AGI before they do, so they have to either stop everyone or not stop themselves. I personally disagree with that take - and, as you note, it's hard to take seriously ethical wrangles from a company that literally sued the government in court to allow their models to be used by Palantir of all people. But if o…

It’s a bit of a self-serving argument, don’t you think? And how does that relate to the ask for oligopoly licensing within global democracy? “We must build the nuclear bomb first in order to make sure no one else builds one.” This the most nonsense, disingenuous argument imaginable.

> It’s a bit of a self-serving argument, don’t you think?

I would call it more of a self-selecting one. Anthropic is basically hiring people with that mentality. I'm pretty sure that most of them do sincerely believe it, too. I'm skeptical about Dario himself though. The man had an opportunity to show moral backbone, and failed to do so; why should I trust him on that again?

> And how does that relate to the ask for oligopoly licensing within global democracy?

They are basically saying that they'll stop if everybody else does, which requires some kind of global enforcement mechanism.

> “We must build the nuclear bomb first in order to make sure no one else builds one.” This the most nonsense, disingenuous argument imaginable.

The difference between nuclear bomb and AGI (as understood by the likes of Anthropic) is that the latter triggers the technological singularity that renders any runner-ups moot. That is, so long as AGI is developed, we're going to get our robot overlords either way, but whoever gets there first gets to define their ethical system. If that is one's perspective, and if one sincerely believes that they are the only ones who can do it right, it's a coherent argument. It's just that the premises are very arrogant.

Your analogy with nukes actually works better for the position that AI development needs to be unconstrained because otherwise we'll lose the arms race to China. That is basically a repeat of https://en.wikipedia.org/wiki/Einstein%E2%80%93Szilard_lette.... I honestly don't know where I am on this. Realistically, if AI is indeed a power multiplier - and with all the recent security stuff it's hard to not see it that way - then an arms race feels inevitable, especially given the current worldwide political situation. I could believe in sincere international cooperation on this back in 1990s, but there's way too much saber rattling all around for it to work (and note that this goes both ways, i.e. China can similarly not be certain that US isn't secretly developing more powerful AI even if we do publicly announce a freeze).

Post reply on HN