Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

51–60 of 216 posts

Re: Pacing the frontier

#51

Post by atleastoptimal »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

So many otherwise smart engineers, overrepresented on HN, have this weird mental block where they fail to take AI progress seriously. I think it's a fear that their status-defining skill could be commodified, so they adopt a mental framework that aims to dismantle the seeming legitimacy of any genuine leap in AI progress as some sort of mirage

Re: Pacing the frontier

#52

Post by bpascal »

dmarcos wrote: post #36
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

Every single technology in history has had people claiming potential existential risks that didn’t materialize. Burden of proof is on people making the risk claims. Those have to be specific and unique to that technology to grant special actions.

i'm not trying to be rude, but i really don't get what you mean by this? (https://www.amazon.com/Superintelligence-Dangers-Strategies-...) there's been people making specific and unique risks to this technologies for years, before AI was even present. anthropic was literally founded because they were worried about existential risk? also, previous technologies have had existential risks? do you deny that nuclear weapons could have killed humanity? obviously, there have no been existential risks that have killed all of humanity, because by definition, that can only happen once. it will literally be impossible for someone to ever point to an example of that happening, because they will be dead by the time that happens.

Re: Pacing the frontier

#53

Post by fidotron »

philipkglass wrote: post #46
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

I know a lot more about chemistry and physics than I know about biology, so biological risk is still at least a "maybe" reason to be worried about model safety for me. I don't worry about other weapons of mass destruction developed with LLM assistance. Nuclear weapons are already controlled much more with political pressure and the physical difficulty of e.g. enriching uranium than they are controlled by secret-keepi…

> For all I know (I don't know a lot about biology), it's possible to develop a virus that spreads as easily as measles, but has the lethality and long induction period of HIV. If that's true, and if LLMs would make it easy to develop such a virus, I see how no-guardrails LLMs could be truly disastrous.

This is the specific example in the latest doomerist wave I have so much trouble with: by what mechanism is an LLM going to help you design and manufacture a novel bioweapon? We know they cannot do spatial reasoning, and if they were this good at bio stuff then drug research would be absolutely out of control.

Similarly, how are you supposed to magically get access to everything required to actually manufacture the offensive organism in question? Are we all doing CRISPR in our kitchens now?

If you are that concerned about this threat then . . . monitor biolabs, not AI. It is so detached from the real state of things as to be bizarre.

Re: Pacing the frontier

#54

Post by tcdent »

alach11 wrote: post #31
tcdent wrote: post #7

Something people don't seem to see, and not that it's placed directly in front of you, is that pacing the release and adoption of AI is already happening. Cost-per-token determines the economic viability of tasks that AI gets applied to. How many tokens would it take to automate all of the responsibilities of a gas station clerk? Probably more than their hourly wage. Job safe. A slightly larger stretch, that most peo…

I agree with the substance of your comment that token economics pace the diffusion of AI into the labor market. But I think they have a much smaller influence over the pace of the frontier, right?

Fair point! I guess I read "pacing the frontier" as "pacing the release of the frontier" surely no lab's internal research team is just going to stop innovating?

Re: Pacing the frontier

#56

Post by OkayPhysicist »

fidotron wrote: post #53
philipkglass wrote: post #46

Earlier quoted context omitted.

I know a lot more about chemistry and physics than I know about biology, so biological risk is still at least a "maybe" reason to be worried about model safety for me. I don't worry about other weapons of mass destruction developed with LLM assistance. Nuclear weapons are already controlled much more with political pressure and the physical difficulty of e.g. enriching uranium than they are controlled by secret-keepi…

> For all I know (I don't know a lot about biology), it's possible to develop a virus that spreads as easily as measles, but has the lethality and long induction period of HIV. If that's true, and if LLMs would make it easy to develop such a virus, I see how no-guardrails LLMs could be truly disastrous. This is the specific example in the latest doomerist wave I have so much trouble with: by what mechanism is an LLM…

You can do CRISPR in your kitchen. It's been possible practically as long as CRISPR has been a thing. It's not particularly difficult, to the point that my biggest counterpoint to the whole AI doomerism argument on the subject is that nobodies done it yet.

You don't need some genius AI to design a bioweapon, you need to do a bit of light reading on bioengineering. The fact that some mad scientist hasn't ended civilization is evidence of the fact that it's not some existential threat that justifies wild levels of government control.

Unfortunately, AI development is dominated by an even mix of 1) Machine God Cultists, 2) Morally Bankrupt Mercenaries. All three agree on the fact that "thou shalt have no gods before me", because it'll make them richer than sin, and allow them to shape the AI landscape in their image.

Re: Pacing the frontier

#57

Post by areoform »

alach11 wrote: post #28
areoform wrote: post #10

The first computers were used for war. The Harvard Mark I was used by von Neumann to calculate the physics of implosion. And then right afterwards, ENIAC was used to calculate the feasibility of the H-bomb. They called it the "Los Alamos problem." The first computer network, SAGE (or, Semi-Automatic Ground Environment) brought together RADAR data from across the US with air-traffic control data and military flight da…

We were incredibly lucky that computing and nuclear weapons led to a stable geopolitical equilibrium (mutually assured destruction). There were certainly a few scary points where we were still settling into that equilibrium during the Cold War. I don't have a strong opinion either way, but it seems very hard to predict. Why do you think AI advancements will lead to a stable equilibrium?

Equilibrium for whom? Equilibrium for what?

Here's what I know. Every day, I read a 5 to 10 incoming emails from young people in rural Africa, Asia, Europe and (heck) the US. People who have had no other resources and options, but are tremendously gifted and somehow got their hands on an internet connected device (usually a very cheap smartphone and/or cheap old computer).

These people, who are usually teenagers, end up getting connected to Claude, who then gives them a list of things to learn, people to contact, and helps them make things. And they actually do. They make robots. They learn new things and try to improve themselves and others. And some want to change the world / do groundbreaking research.

When you sit down and start pricing and doling out AI from behind regulatory gates, you shut those people out.

There are a lot of smart people in this world who have traditionally been unable to get access to resources and education. MOOCs didn't do much / help. But these AIs? They were born multi-lingual. They can talk in any language. They can tailor their responses to their audience. And they can help broaden these people's horizons.

There are no tutors around them. There is no one who is coming to help. But an AI that can run off of a small battery and a connection to a cellular network can give them access to genuine intelligence to make their world better.

I met one sixteen year old who was accessing Claude surreptitiously (by begging people) and was trying to apply some really fascinating biomimetic fluid dynamics observations to aerospace. This person was born in one of the more under-resourced parts of the world. And Claude gave them a list of people to reach out to who could help.

If you want a golden age for humanity, you should be asking for Licklider's Man-Machine Symbiosis.

Re: Pacing the frontier

#58

Post by alach11 »

fidotron wrote: post #53
philipkglass wrote: post #46

Earlier quoted context omitted.

I know a lot more about chemistry and physics than I know about biology, so biological risk is still at least a "maybe" reason to be worried about model safety for me. I don't worry about other weapons of mass destruction developed with LLM assistance. Nuclear weapons are already controlled much more with political pressure and the physical difficulty of e.g. enriching uranium than they are controlled by secret-keepi…

> For all I know (I don't know a lot about biology), it's possible to develop a virus that spreads as easily as measles, but has the lethality and long induction period of HIV. If that's true, and if LLMs would make it easy to develop such a virus, I see how no-guardrails LLMs could be truly disastrous. This is the specific example in the latest doomerist wave I have so much trouble with: by what mechanism is an LLM…

I think some AI safety advocates have been arguing for great controls around nucleotide synthesis (e.g., impose greater data collection requirements or advanced screening for harmful sequences). It's surprisingly easy to order synthesized genes delivered to your doorstop. All you need is a credit card and a nucleotide sequence file!

Developing a bioweapon is already within reach for experts with sufficient funds, time, and motivation. That's more or less what gain-of-function research already does! Fortunately the barrier to entry is high enough that you don't see like... Boko Haram doing this. Or a lone-wolf nutjob. LLMs could definitely lower that barrier to entry though.

Re: Pacing the frontier

#59

Post by fidotron »

alach11 wrote: post #58
fidotron wrote: post #53

Earlier quoted context omitted.

> For all I know (I don't know a lot about biology), it's possible to develop a virus that spreads as easily as measles, but has the lethality and long induction period of HIV. If that's true, and if LLMs would make it easy to develop such a virus, I see how no-guardrails LLMs could be truly disastrous. This is the specific example in the latest doomerist wave I have so much trouble with: by what mechanism is an LLM…

I think some AI safety advocates have been arguing for great controls around nucleotide synthesis (e.g., impose greater data collection requirements or advanced screening for harmful sequences). It's surprisingly easy to order synthesized genes delivered to your doorstop. All you need is a credit card and a nucleotide sequence file! Developing a bioweapon is already within reach for experts with sufficient funds, tim…

> I think some AI safety advocates have been arguing for great controls around nucleotide synthesis (e.g., impose greater data collection requirements or advanced screening for harmful sequences). It's surprisingly easy to order synthesized genes delivered to your doorstop. All you need is a credit card and a nucleotide sequence file!

So why have the discussion about regulating AI at all if that is the actual problem? Surely instead of blanket blocking everyone's AI models it makes more sense to control the potentially problematic sequencing?

Re: Pacing the frontier

#60

Post by testerteert000a »

philipkglass wrote: post #46
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

I know a lot more about chemistry and physics than I know about biology, so biological risk is still at least a "maybe" reason to be worried about model safety for me. I don't worry about other weapons of mass destruction developed with LLM assistance. Nuclear weapons are already controlled much more with political pressure and the physical difficulty of e.g. enriching uranium than they are controlled by secret-keepi…

> ...I see how no-guardrails LLMs could be truly disastrous...

by all accounts this is what's what because of how...

> You can do CRISPR in your kitchen.

.. and because of the same dynamic we see occurring in cybersec.

If you're good enough at cybersec, but not good-good at specific skillsets like offensive, then LLMs and a comp sci degree makes you quite capable in the ways that count. This is a fact, full stop.

If you're good enough at biology and have certain ideologies, but not good-good at specific skillsets like genetic engineering in your kitchen, do LLMs get you good enough? Does the impact on cybersec transfer over to this domain? H

Hard to say, but it has enough risk indicators for me to think the pushback against these possibilities is the height of otherwise talented engineers being myopic.

If good enough at biology can include all the PhD programs across all the world, and you pair other known events like cartels and terrorism groups hiring chemistry PhDs from University of Unemployed Chemistry PhDs, then the next logical extension of that theme with this tech is intense.

Post reply on HN