Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

101–110 of 216 posts

Re: Pacing the frontier

#101

Post by sm-silversight »

OkayPhysicist wrote: post #56
fidotron wrote: post #53

Earlier quoted context omitted.

> For all I know (I don't know a lot about biology), it's possible to develop a virus that spreads as easily as measles, but has the lethality and long induction period of HIV. If that's true, and if LLMs would make it easy to develop such a virus, I see how no-guardrails LLMs could be truly disastrous. This is the specific example in the latest doomerist wave I have so much trouble with: by what mechanism is an LLM…

You can do CRISPR in your kitchen. It's been possible practically as long as CRISPR has been a thing. It's not particularly difficult, to the point that my biggest counterpoint to the whole AI doomerism argument on the subject is that nobodies done it yet. You don't need some genius AI to design a bioweapon, you need to do a bit of light reading on bioengineering. The fact that some mad scientist hasn't ended civiliz…

Can you point me at this light reading? I don't wanna do CRISPR in my kitchen, but itd be cool to know more about it. I thought I needed to be basically pre-med to grasp it.

Re: Pacing the frontier

#102

Post by txrx0000 »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

There is constructive discourse, but this kind of commentary doesn't get to the top:

https://news.ycombinator.com/item?id=49078376

https://news.ycombinator.com/item?id=49077753

It's not a given that AI poses bio/political/cyber/economic risk in a way that warrants government intervention. Discourse should first revolve around whether it makes sense for the government to step in at all.

Re: Pacing the frontier

#103

Post by aesthesia »

dmarcos wrote: post #93
aesthesia wrote: post #85

Earlier quoted context omitted.

How many of those concerns came from the people building the technologies? John Carmack wasn't up in arms about how Doom was going to destroy society. Charles Dickens didn't think novels rotted the brain. Johannes Gutenberg wasn't concerned about the effects of a free press.

None of the previous concerns were true and nothing indicates current ones are. It’s extrapolation, hypothetical apocalyptic imagined scenarios and piles of assumptions on how the technology could evolve. What makes even less credible in the AI case is that there’s financial / power to be gained by those making the claims. And to your point. Don Quixote, the first modern and greatest novel of all times, is about a gu…

I mean, Cervantes was parodying medieval romances. That's what Don Quixote was reading, not modern novels.

The point is, it should give you at least some pause when the people most familiar with a technology are among those making the strongest statements about its potential dangers. Extrapolation has worked extremely well in predicting what AI will be capable of over the past 5 years; how confident are you that it will fail before we reach apocalyptic scenarios?

Re: Pacing the frontier

#104

Post by txrx0000 »

atleastoptimal wrote: post #51
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

So many otherwise smart engineers, overrepresented on HN, have this weird mental block where they fail to take AI progress seriously. I think it's a fear that their status-defining skill could be commodified, so they adopt a mental framework that aims to dismantle the seeming legitimacy of any genuine leap in AI progress as some sort of mirage

It's ironic that the top reply to a comment about quality of discourse on this site is an ad hominem. Most people here take AI progress very seriously judging by the number of posts on the front page about AI at any given time. And I think some engineers even explicitly want to commodify their "status-defining skill":

https://nitter.net/__tinygrad__

Re: Pacing the frontier

#105

Post by yellowapple »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

> It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest.

It's very easy to make those dismissals because of how transparently obvious it is that self-interest is a significant factor, no matter how loudly they insist otherwise.

> If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic risks, surely a joint petition like this is a good thing?

Similar attitudes hindered nuclear energy research and deployment for decades, in turn kneecapping our species' ability to obsolesce fossil fuels and avoid the global climate catastrophe we've now made inevitable.

Re: Pacing the frontier

#106

Post by bpascal »

gr_norm wrote: post #96
marcus_holmes wrote: post #86

Earlier quoted context omitted.

You're right, in that we would have said that. I wouldn't frame it as moving the goalposts in a bad way, though. We understand more now than we did then, so we have a better idea of what the actual goal is, and moving the goalposts to frame that goal is more rational than insisting the goalposts stay where they were ten years ago even though the entire pitch has changed since then. I'm straining the analogy, obviousl…

Moving the goalposts, so to speak, is how science works! We must of course update the things we think based on an improved understanding of how the world works. Only the deeply incurious could consistently demand that one adhere dogmatically to a prior way of understanding the world in the face of changing knowledge. It's important not to treat science as a game to be won (of which 'goalposts' are evocative), but as…

what do you mean by 'general intelligence'? i definitely agree that in comparison to humans, they have some definite weaknesses. however, i think they also have definite strengths. and i don't think their current weaknesses imply that much about a potential singularity or ai risk.

Re: Pacing the frontier

#107

Post by ascorbic »

dmarcos wrote: post #93
aesthesia wrote: post #85

Earlier quoted context omitted.

How many of those concerns came from the people building the technologies? John Carmack wasn't up in arms about how Doom was going to destroy society. Charles Dickens didn't think novels rotted the brain. Johannes Gutenberg wasn't concerned about the effects of a free press.

None of the previous concerns were true and nothing indicates current ones are. It’s extrapolation, hypothetical apocalyptic imagined scenarios and piles of assumptions on how the technology could evolve. What makes even less credible in the AI case is that there’s financial / power to be gained by those making the claims. And to your point. Don Quixote, the first modern and greatest novel of all times, is about a gu…

None of the previous concerns were true, largely because they came from laypeople who didn't understand the technology. A prominent exception is nuclear weapons, where their creators did worry their creations could end up destroying the world. I think most people would agree those concerns were justified, and it's not any lack of capability that means we're still here.

The motivations aren't unique here. Restrictions on the development of almost any technology benefits incumbents by providing a moat.

Re: Pacing the frontier

#108

Post by BugsJustFindMe »

fidotron wrote: post #53
philipkglass wrote: post #46

Earlier quoted context omitted.

I know a lot more about chemistry and physics than I know about biology, so biological risk is still at least a "maybe" reason to be worried about model safety for me. I don't worry about other weapons of mass destruction developed with LLM assistance. Nuclear weapons are already controlled much more with political pressure and the physical difficulty of e.g. enriching uranium than they are controlled by secret-keepi…

> For all I know (I don't know a lot about biology), it's possible to develop a virus that spreads as easily as measles, but has the lethality and long induction period of HIV. If that's true, and if LLMs would make it easy to develop such a virus, I see how no-guardrails LLMs could be truly disastrous. This is the specific example in the latest doomerist wave I have so much trouble with: by what mechanism is an LLM…

> We know they cannot do spatial reasoning

Everything else aside, and regardless of whether that claim is currently true, I would ask you to be aware that that statement requires a way of thinking that ignores predictable future developments. Whatever the capability of any systems currently released, there will be more capable ones tomorrow, and the most you can say is that we have not mastered training spatial reasoning yet, not that we can't or won't.

> Are we all doing CRISPR in our kitchens now?

https://www.the-odin.com/crispr-kit/

Re: Pacing the frontier

#109

Post by ajyoon »

intorio wrote: post #69
ajyoon wrote: post #65

Earlier quoted context omitted.

The entire point of the statement is to start international coordination.

The problem is how do you enforce coordination? This isn't like nuclear testing that can be detected around the world on seismographs or from satellites in space. AI research happens in benign looking buildings that could be doing a hundred different non-AI workloads, much of it things that industry or governments would not want 3rd parties prying into. There is too much to potentially gain to trust coordination.

There is a lot of research out there with answers to your questions. GPU supply chains are extremely constrained, making it a surprisingly tractable problem at least for now.

Re: Pacing the frontier

#110

Post by potsandpans »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

What happens when a country doesn't agree to play along with this little decel game?

The genie is already out of the bottle and now a bunch of people who helped release it think that the collective powers of the world are just going to put it back in.

How far are the saftyists willing to go? Bombing data centers? Starting wars? Embargos?

It's a terrible dilemma to be in. If the threats are real, it gives all the more incentive for some actor to race and try to grab it first. If not, then any punishment isn't proportional. Explain to a nations citizens under a 10 year embargo, "sorry about your quality of life, we all just got a little spooked by science fiction for a second."

Folks try to reach for bad analogies: nuclear bombs and biological weapons. But conveniently omit that by the time the regulations took place for these things we had unequivocal proof of their destruction, scope and scale.

So, what then? We're supposed to collectively stop innovating and put the keys to the kingdom in the hands of a few capitalists, a corrupt is administration and the ccp?

I reject that these agreements should take place, on the grounds that the proposal is delusional and not founded in reality.

Post reply on HN