Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The moments that could have accidentally ended humanity

bbc.com

Post reply on HN
Print view View original

11–20 of 78 posts

Re: The moments that could have accidentally ended humanity

#11

Post by tuatoru »

sarbaz wrote: post #3

The article has some great examples of risks that were hyped out of proportion to their true severity. Should these things be considered? Yes. But only a little. If you halt programs completely for tail risks you'll never get anywhere. Modern examples, IMO, are: - Kessler Syndrome - Trying to prevent an asteroid hitting the earth - Nuclear war making the entire earth uninhabitable

I think you mean "probability" rather than severity. The severity of setting the atmosphere on fire is extreme, but the probability turned out to be vanishingly small. Or perhaps you mean risk, probability times severity. Which also turned out to be negligible for the events in the article.

Some other hyped risks: -

- Artificial General Intelligence

- CRISPR gene editing

- Gain-of-function work with viruses

I have no way of assessing the risks, and there is a lot of hyperventilation in some circles

Re: The moments that could have accidentally ended humanity

#12

Post by zh3 »

Animats wrote: post #4

Some big threats: - Something with the spread rate of COVID-delta, and a high lethality rate after a long incubation period. - Enriching uranium in a rather small facility. This may already have happened and been kept quiet. Laser enrichment was talked about a lot in the early 1990s, and then suddenly, after some announcements from Lawrence Livermore, things got much quieter.[1][2] As high-powered lasers get better,…

Long conjectured, some disillusioned uni student with the latest tech combining the R number of measles withe the lethality of ebola (and a dash of HIV on the side).

As a twist, "The giving plague" by David Brin is an interesting (short) take on it.

Re: The moments that could have accidentally ended humanity

#13

Post by whateveracct »

no_wizard wrote: post #2

It’s moments that I read articles like this that make me wonder why I’m putting so much energy into making a better SPA or trying to engineer solutions to faster more performant web animations. I feel like I’m not using any of my technical acumen to advance human civilization but instead to line my own coffers and those of the corporation I work for. With that said, it is always good to keep in mind just how fragile…

I mean .. do you enjoy the work?

I write Haskell, I make computer games, I make web apps. Mostly because it's fun and satisfying. I'm also quite good at it and it comes easily to me.

I remember when I decided to go into engineering, a peer of mine from high school said "whateveracct, you're top of your class. Why aren't you going into medicine in order to do something more Worthwhile with your life?"

Stuck with me ever since. I was repulsed by the mindset but I couldn't word why at the time. I later realized that it smelled of a deeply nihilistic (as in Nietzsche's ideas) view of the world. Ressentiment comes to mind.

Spending my conscious hours working with computers is a less nihilistic use of my time. I am not deferring this life's happiness and agency in order to "have made an impact" when my life is over and useless to me.

If you want more philosophy, consider Plato's Republic. An ideal society doesn't necessarily have everyone doing Most Important and Dire Work. It doesn't even have them doing what they're most "skilled" at! Instead, it has everyone living in alignment with their souls' desires and preferences. (e.g. A frail person with a Warrior's Soul should be a soldier before a strong person with an Artisan's Soul.)

Re: The moments that could have accidentally ended humanity

#14

Post by stank345 »

no_wizard wrote: post #2

It’s moments that I read articles like this that make me wonder why I’m putting so much energy into making a better SPA or trying to engineer solutions to faster more performant web animations. I feel like I’m not using any of my technical acumen to advance human civilization but instead to line my own coffers and those of the corporation I work for. With that said, it is always good to keep in mind just how fragile…

I know how you feel. You might be interested in the 80,000 hours podcast as a way to gain a better understanding of the problems humanity faces: https://80000hours.org/podcast/

In terms of what to _do_ about it as a software developer, I'm still trying to figure that one out. I currently work at a BCorp which tends to make me feel better about the work I'm doing which at a minimum isn't doing harm to the world. You could try looking for a meaningful job at https://techjobsforgood.com/

Re: The moments that could have accidentally ended humanity

#16

Post by perihelions »

Does SETI not count? Easy to imagine the first received ETI signal being some world-ending infohazard.

[edit]: Targeting humans: a viral ideological, philosophical, or religious meme that causes us to self-destruct. A technological gift with insidious Trojan Horse functionality (a biotechnological machine with subtle side effects; a physics device that touches physics we haven't discovered). Irresistible instructions on how to join the friendly galactic internet -- which actually route to a paranoid deviant ETI that destroys anything that transmits (ala Dark Forest hypothesis).

Targeting machines: exploting a buffer overflow in the Allen Array's signal processing pipeline to upload onto this planet a self-replicating superintelligent AI.

Targeting the superior chthonic race living in the Earth's mantle that we don't know about: friendly instructions on how to terraform the terrestrial surface and become a spacefaring species.

Re: The moments that could have accidentally ended humanity

#17

Post by dharma1 »

Feels like gene editing is at a point where more frequent lab escapes of self-replicating airborne pathogens is just a question of time. Not to mention the speed of progress in biotech and synthetic biology means soon this stuff can be done in a garage.

Beyond pathogens, perhaps when life (inevitably) learns to change the code that makes up life, the recursion leads to implosion soon after

Re: The moments that could have accidentally ended humanity

#18

Post by forty »

hn_throwaway_99 wrote: post #6
Animats wrote: post #4

Some big threats: - Something with the spread rate of COVID-delta, and a high lethality rate after a long incubation period. - Enriching uranium in a rather small facility. This may already have happened and been kept quiet. Laser enrichment was talked about a lot in the early 1990s, and then suddenly, after some announcements from Lawrence Livermore, things got much quieter.[1][2] As high-powered lasers get better,…

> Something with the spread rate of COVID-delta, and a high lethality rate after a long incubation period. I sometimes think how "lucky" (obviously, luck is relative) humanity is that HIV is an STD and not an airborne virus with the transmissibility of Delta.

For some reasons I feel HIV treatments would have been found much more quickly had it been the case. Just like I'm pretty sure mosquitos would have been eradicated (or another solution would have been found) if it had killed so many people in developed countries.

Re: The moments that could have accidentally ended humanity

#19

Post by eloff »

sarbaz wrote: post #3

The article has some great examples of risks that were hyped out of proportion to their true severity. Should these things be considered? Yes. But only a little. If you halt programs completely for tail risks you'll never get anywhere. Modern examples, IMO, are: - Kessler Syndrome - Trying to prevent an asteroid hitting the earth - Nuclear war making the entire earth uninhabitable

> If you halt programs completely for tail risks you'll never get anywhere.

If your tail risk is the end of civilization then it doesn't matter how small the probability. You'd be fucked with certainty on any long enough timeframe.

Some tail risks are to large to take. Eventually your number comes up.

Post reply on HN