Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone should slow down AI development except for me

xeiaso.net

Post reply on HN
Print view View original

161–170 of 237 posts

Re: Everyone should slow down AI development except for me

#161

Post by ElProlactin »

hn_submit wrote: post #151

This is exactly the reason Anthropic's call to slow down A.I. development will fail. Shareholders will be loath to accept that the company loses its lead and a commensurate sharp decline in the company's share price. Moreover, it could lead China to catch up and eventually proclaim that it has nosed ahead of the U.S. in the field. The U.S. Government won't allow that to happen.

> The U.S. Government won't allow that to happen.

Like the US government wouldn't allow Iran to close the Strait of Hormuz? Or a bunch of sandal-wearing Islamists to take control over the Red Sea coastline?

The US government is not omnipotent. To the contrary, it's increasingly impotent.

China has somewhere around 200x more shipbuilding capacity than the US today. It has more shipbuilding capacity in one shipyard that the US has in all of its shipyards. China is already a force in AI and there is nothing the US government can do to stop it at this point.

Re: Everyone should slow down AI development except for me

#162

Post by fc417fc802 »

metalcrow wrote: post #148
fc417fc802 wrote: post #137

Earlier quoted context omitted.

What I'm asking is, why should we expect that to effectively further the end goal? You're simply asserting that it will ultimately do so. Given the efficiency gains we've seen it seems to me that the situation has shifted from being analogous to producing nuclear weapons to producing something much closer to small arms.

To further the metaphor, didn't the ban on nuclear weapons research work? We don't have pocket nukes, and there's very little risk of random countries acquiring their own due to the amount of work involved. To be concrete to AI: if we stop work on the frontier, the best we can do is make the current frontier easier to get to and more accessible. And while current frontier is strong, it's not world-changing so. The on…

Not in the way you seem to be thinking, no. Nuclear weapons can be banned because they are prohibitively expensive to research and build. You can't reliably hide a nuclear program.

In contrast, rewind to the early 1800s and there is zero hope of a ban on the R&D of small arms being effective in the long run. The only thing it might maybe ensure is that no legitimate actors that fall under your jurisdiction are involved in it.

Basically I think that current trends point to an eventual situation where world changing research doesn't require anything more than consumer level compute. Pandora's box has already been opened.

> We don't have pocket nukes

Bit of a tangent but we do, actually. At least depending on if you consider an artillery shell to be pocket sized. https://en.wikipedia.org/wiki/Nuclear_artillery

This one is a bit long to fit in a pocket but you could certainly sling it over your back. https://en.wikipedia.org/wiki/W45_(nuclear_warhead)

Re: Everyone should slow down AI development except for me

#163

Post by rtpg »

lukan wrote: post #34
ranger_danger wrote: post #31

Earlier quoted context omitted.

I figured they're just admitting AI models have plateaued and are coming up with some fake story about self restraint so they don't lose VC money

Not sure about the level of irony here, but I keep hearing models have plateaued since a while now, but I keep being impressed with the latest model performance.

I'll take the opposite here. If someone put in frontier AI models from like .... last june I guess? in a box and let me run it with "decent" token throughput I would be happy.

I think it's worth acknowledging that the power of LLMs at this point is not really so much in the smarts, but in the coordination and the surrounding harness tech. "Written english" turning into sequences of commands[0]. The whole agentic "stuff" in general. Tools + coordination is the superpower. The reasoning... it doesn't have to be _that_ good for the rest of the stuff to work. On good codebases and infra, at least.

And I say this as someone who really would rather most of this stuff disappear!

[0]: programming is obviously text to commands, but there's a loooooooot of futziness that LLM reasoning has let us remove in some flows

Re: Everyone should slow down AI development except for me

#164

Post by fc417fc802 »

sm-silversight wrote: post #143
m4rtink wrote: post #88

Earlier quoted context omitted.

In some cases, such as space exploration I don't really care who does it but that it happens - sure, would be nice if my favorite power block did it but I will still celebrate it when someone else achives it. Can still be a powerful motivation, to make sure that next time, it yous your camp that scores the next milestone, like an orbital elevator or fox ears, for example.

Fox ears?

If what you want is fox ears does it matter to you which country manages to come up with the biomedical procedure to give them to you? Ditto for enhanced eyesight, a replacement liver, or whatever it is you're after.

Re: Everyone should slow down AI development except for me

#165

Post by dvt »

Obligatory "what's Lygma?" But in all seriousness, this AI doomerism has reached a comedic inflection point. A few years ago, GPT-2 was too dangerous to release, then some Google weirdo said Gemini had a soul or something, then Mythos-tier became a meme, then some kid quit and went on FOX News talking about Skynet, and now Amodei and Altman are both on the "we need to slow down" train again; this is after we've already been down that road and Claude was banned overseas; wait, actually is that ban still around? Honestly who gives a fuck at this point.

It's all theatre. OpenAI and Anthropic will most likely go bust—or, more realistically sold for parts—, and they absolutely should for stealing my (books I wrote, blog posts, etc.) and many others' intellectual property. We're reaching a point where models are becoming commodetized and I'm 100% convinced the next move will be a sort of "software layer" on top of these reasoning systems which will be the actual revolution. The model itself won't be that interesting anymore, it's all the work that goes around it that makes it worthwhile (kind of what computers and phones are today; chips are amazing, but the software is really the magic).

The only scary part is that the boomers in Congress might actually believe these nerds, but seeing how Big Tech approval ratings are grazing the levels of Big Tobacco in the 90s, I don't think we have much to worry about.

Re: Everyone should slow down AI development except for me

#166

Post by red75prime »

nullbio wrote: post #39

These people are so fucking exhausting. And then Dario wants to recommend METR as the "independent evaluator" while he stacks their org full of ex-Anthropic (aka, secretly still on the Anthropic payroll with huge equity) employees. "We'll give them a desk, an office, a work laptop, ..." Fucking make it less obvious. I kind of hope the govt steps in at this point and says "Anthropic, you wanted regulation? We've creat…

Oh! A testable prediction. Here's mine: there will surely be a lot of politicking around who qualifies to be the independent evaluator, but they'll agree on something because they are really scared.

Re: Everyone should slow down AI development except for me

#167

Post by SV_BubbleTime »

amazingman wrote: post #141
aorloff wrote: post #123

Earlier quoted context omitted.

The very best outcomes involve us intentionally and collectively turning our attention away from this technology.

That is pure fantasy.

Watch, very soon this is going to politically divide. They’re sowing the seeds right now with “antiai” campaigns.

The right will for a change be for it but will be able to be talked into regulation because you know “small government” and all only when convenient. The left will bitch and moan about fairness and copyright and automation and UBI, they’ll be the doomers and worldwar chicken littles. The Uniparty will be for it, but only against the public having anything good.

What will be funny to me is that normal tech people outside the frontier model companies are about to find themselves without a home on this topic.

Re: Everyone should slow down AI development except for me

#168

Post by visarga »

nullbio wrote: post #157
konmok wrote: post #122

Earlier quoted context omitted.

> The latest studies demonstrate model collapse is not a given Which studies? [edit: I'll assume you mean these two given by @dorolow: https://arxiv.org/abs/2404.01413 https://arxiv.org/abs/2406.07515 ] > It can't be used as the -only- data source of course, but that's not how it is being used Right, so human data creation would also have to scale up exponentially, and that's not gonna happen. > because there's no di…

> Right, so human data creation would also have to scale up exponentially, and that's not gonna happen. It doesn't need to. We're not even close to exhausting the useful synthetic data within the human data we have, let alone all of the new data that is being created. > I mean, that's obviously false, otherwise model collapse wouldn't exist. The difference is statistical, but it's there. It's not. It's just bytes of…

[dead]

Re: Everyone should slow down AI development except for me

#169

Post by visarga »

nullbio wrote: post #118
ranger_danger wrote: post #101

Earlier quoted context omitted.

Can you provide sources for these claims?

What claim do you have a problem with? There are plenty of research papers on synthetic data that show its value, do a search on arxiv for "synthetic data". There are plenty of open-source post-training pipelines that incorporate synthetic data. As for the claim about accelerating the progress of hardware or materials science, I've seen quite a number of news articles from teams at universities using AI in their work…

It's not even synthetic data as such - often it is environments. So the models create their own data solving tasks in generated environments. I am making one such environment for computer use agents, 600 tasks, each of them a mini app.

Re: Everyone should slow down AI development except for me

#170

Post by lacy_tinpot »

AngryData wrote: post #150
amazingman wrote: post #141

Earlier quoted context omitted.

That is pure fantasy.

Why? There are numerous technologies we have ignored or abandoned for numerous reasons. Many of the benefits of AI is "requires less humans", but in an age where we question what work people could possibly do in the future, human labor isn't that hard to find. If the amish can do what they do for whack religious reasons, people could manage it with ai for cultural and social reasons. And I never saw an amish person s…

You can't just stop AI without also stopping progress in computer hardware. You have to do both.

The real implication of what they're saying extends beyond just AI.

You have to stop the entire compute stack in order to prevent or slow down AI progress.

Post reply on HN