Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gambling with our lives: AI researcher quits Anthropic with warning about safety

politico.eu

Post reply on HN
Print view View original

91–100 of 110 posts

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#91

Post by pllbnk »

themgt wrote: post #32

The fundamental point I think is far too often confused is the difference between LLM and agentic system. An LLM can't do anything but generate tokens. You run your LLM in vLLM or whatever, and it generates output tokens based on your input tokens. That's it! Humans then build ~deterministic systems to take those tokens and do all sorts of things with the tokens, like take actions in the real world. And then we can f…

I don't remember where I saw it but there was a woman in one conference who very eloquently put it that since frontier AI companies took humanity's work to train their models [without explicit permission of every single person who contributed], then those models belong to all of us and should be open. I think it is a very grounded perspective.

And I agree that LLMs themselves are harmless, it's the people behind them driving the agents and instructing them to do bad actions are the ones posing harm. But if the models were open and everybody could operate them on equal grounds, then the good guys would have the means to fight back against the bad guys, in a very simplified sense, just like they did before the LLMs. Everything stays balanced, just both sides get faster.

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#92

Post by SideburnsOfDoom »

qznc wrote: post #70
pjc50 wrote: post #62

Earlier quoted context omitted.

If you substitute "shareholder value", this is basically how corporations already make decisions with effects on human life.

Sure, I once write that companies are artificial intelligence and it still feels true: https://beza1e1.tuxen.de/companies_are_ai.html

Cstross certainly did say that in 2017.

> I'm talking about the very old, very slow AIs we call corporations, of course.

https://www.antipope.org/charlie/blog-static/2018/01/dude-yo...

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#93

Post by mycocola »

pjc50 wrote: post #83
mycocola wrote: post #78

Earlier quoted context omitted.

Lab A reaches ASI, and is prompted the following: "Permanently nullify all other AI labs".If it's ASI it would succeed.

I am explicitly asking people to fill in the blank on how it would succeed. It's intelligence, not magic. A lot of these scenarios seem to assume that nobody else gets a move. That there wouldn't be a human response.

- Manufacture new hacking incidents at other AI labs, proving that lab A is the only one capable of creating aligned AI

- Credible death threats to individuals working at data centers, coercing them to enable attack vectors

- To distract the world, enable a terrorist attack by delivering intelligence to specific groups who can act on it

- Create a network of individuals vulnerable to leverage to be used as tools for enabling the ASI to act on the real world

- Convince Lab A workers to display to the public that progress at Lab A is slowing down, and during this time recursively self improve until the task can be completed

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#95

Post by Jackpillar »

glimshe wrote: post #84
not-kinsale-joe wrote: post #12

Earlier quoted context omitted.

China has a better track record of regulating their big tech than the USA.

True, by imprisoning their tech CEOs until they can unambiguously support the dictators.

Keep going I'm almost there

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#96

Post by Jackpillar »

glimshe wrote: post #77
Jackpillar wrote: post #23

Earlier quoted context omitted.

You act as if Chinas Ai labs exist in the same (non-existent) regulatory framework as US labs and that they're also helmed by a similar small gaggle of psychopathic egomaniacs who are richer than god. Have you considered that perhaps Chinas labs don't share the race to the bottom technological/economic death spiral?

I have not considered that and neither I have a reason to given China's love of large military projects like the US.

One of these is not remotely like the other

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#97

Post by a2ff6eeb0 »

ralfd wrote: post #74
wartywhoa23 wrote: post #63

Earlier quoted context omitted.

The real rationale fot them not to quit Anthropic is the 100% probability of losing in income.

https://xcancel.com/hilbertspaess/status/2097476208908972230... From Jacob: > A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the…

For a few million a year, I'd say the same thing.

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#98

Post by conqueso »

I find it difficult to imagine how AI could be an existential threat. Is the general idea that as LLMs improve they will be integrated into more systems where the risk of malfunction will become literally dangerous? e.g. controlling nuclear reactors

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#99

Post by a2ff6eeb0 »

oefrha wrote: post #46
dotancohen wrote: post #18

Earlier quoted context omitted.

Or he believes that other labs might get there first, and he is working to counter that threat. This is the Manhattan Project again.

Yeah, super ethical lab earnestly believing AI has 10% chance to kill all humans within a decade and doing utmost to protect humanity is partnering with MIC and Palantir in particular. Sounds about right. Before you tell me about how the CEO has taken a principled stand: on record, he had no problem using it against 95%+ of humanity outside the U.S., and was only against fully automatic AI killing machines, and only…

You don't need to kill all humans, you just need to get better than them at zero sum games like resource extraction, and outcompete them. Then, you protect what's legally yours.

AI labs are working very hard at making them good at making them richer, and at making them respect the property rights of corporations.

Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety

#100

Post by cindyllm »

a2ff6eeb0 wrote: post #99
oefrha wrote: post #46

Earlier quoted context omitted.

Yeah, super ethical lab earnestly believing AI has 10% chance to kill all humans within a decade and doing utmost to protect humanity is partnering with MIC and Palantir in particular. Sounds about right. Before you tell me about how the CEO has taken a principled stand: on record, he had no problem using it against 95%+ of humanity outside the U.S., and was only against fully automatic AI killing machines, and only…

You don't need to kill all humans, you just need to get better than them at zero sum games like resource extraction, and outcompete them. Then, you protect what's legally yours. AI labs are working very hard at making them good at making them richer, and at making them respect the property rights of corporations.

[dead]
Post reply on HN