721–730 of 1001 posts
He resigned and now what? There are thousands willing to do his role, and many labs are competing in that race. His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.
Earlier quoted context omitted.
This is not correct. Please go through each one again, taking special note of global warming and nuclear weapon development .
Why do you think global warming could kill literally every human being? What's the scenario in which that happens?
It's silly to treat these things as separate categories as if only one will happen at a time. These issues are happening, and are going to happen all at once. That is the fundamental challenge. AI plays into that because it can make wars and nuclear exchanges so much more effective.
If an AI tells the President the USA can survive China's nuclear barrage mostly unschathed, perhaps he'll press the button...
Earlier quoted context omitted.
I still have to correct Claude on very basic misconceptions whenever I get it to code shit. Sometimes it gets wrong things that I had spelled out already. It may be the Doomsday machine, but it is a very silly one. If it kills humans it will do so by mistake. "You are completely right! Humans cannot breathe sulfur dioxide! My mistake, and I take complete responsibility"
> I still have to correct Claude on very basic misconceptions whenever I get it to code shit. Can you give a simple example? I would have agreed 2 years ago, but it's extremely rare I see a frontier model making a silly mistake these days.
It was to build an extremely simple job using an internal framework to walk through a table and log the ids os some records that have a certain scenario.
There's a ton of jobs exactly like this in the codebase, and the framework code is in the codebase as well.
It was so silly I even thought of writing it myself, probably took me longer to steer claude to do it for me.
Anyway, it refused to use a method from the framework to retrieve the parameter as a list, it wanted to retrieve it as a string and parse the commas. I had spelled out in the initial prompt what method it should use.
I really don't like Claude much. Frontier my ass.
“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
In 1900, there was no evidence of the kind you seek that lighter-than-air flight was possible. Good thing some people were foolish enough to ignore you then. Why you would cordon yourself off from the kind of reasoning that predicts legitimately new things, rather than just scaled up versions of the present?
Not to put too fine a point on it, every example you give is based in concrete evidence, some try to think through the implications farther than others, resulting in larger or smaller error bars around the conclusions.
Let me back up though. Maybe in the collaborative human effort here, we are better off having very concrete thinkers, like you seem to be, along with abstract thinkers like the "divorced-from-reality" Rationalists. Personally, I wish we had a stronger culture of collaboration and assuming good-faith and competence in our peers. In my experience, blanket dismissals are very rarely grounded in reality and mostly grounded in fears.
It's as if two private companies are each building increasingly large nuclear bombs, both saying they'd love to stop but it would be unsafe to let any one company be in control of the nukes.
That's basically the reasoning behind MAD, and it checks out? See what happened to any nation that ever gave away their nukes.
Earlier quoted context omitted.
I agree it’s not likely , but I really don’t see how one can dismiss the possibility of immense danger outright. I can think of some scenarios that are not far off from current capability and I wouldn’t be too surprised if the first one occurred within ~1 year from now if there are more “ambitious” unmonitored training runs like OpenAI’s: Example 5: An AI given a goal within a tightly-constrained sandbox figures the…
Example 6 is a good one. Iran attacked water infra in the US recently and maybe they would have done a “better” job (from their point of view) had they used Fable. The “worst case” with 6 is potentially very bad but I think we are currently using advanced AI models to harden systems and patch vulnerabilities more aggressively than anyone is trying to bring down the whole power grid (for example). I think it’s a poten…
If the answer is "they'd at least try to nuke the country that did it in response", then once again, LLMs are not the main threat.
History did put their theory to test after all. It didn't work.
Earlier quoted context omitted.
I still have to read a compelling argument on how AI will "extinct" humanity.
There are future scenarios in which swarms of drones hunt down every single one of us, but why would they? And currently it makes absolutely zero sense because they are completely dependent on us. And even if not, it would be like humanity going on a mission to kill every single cat on Earth. It makes zero sense.
Even if it wouldn't want to do this at first, the fact that it'd have the capability to seems bad.
No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.
> See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. The difference is that none of those are available to individuals.