Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The hacker sent by Anthropic to calm the government's nerves about AI safety

wsj.com

Post reply on HN
Print view View original

81–90 of 131 posts

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#81

Post by anon373839 »

tychez wrote: post #74
teaearlgraycold wrote: post #25

Earlier quoted context omitted.

This is 99% petty drama between the US government and Anthropic and 1% actual safety concerns.

I just find this idea bizarre. This bizarre social media meme that AI just performative when Opus 4.8 is just unbelievably good. As if it is so difficult to believe that a more capable model than Opus 4.8 might actually be dangerous and not just entirely a marketing stunt like a person waving to cars in a chicken outfit. I think it is really this strange form of socialization that people have internalized an anonymou…

> … when Opus 4.8 is just unbelievably good. As if it is so difficult to believe that a more capable model than Opus 4.8 might actually be dangerous

It’s funny, but this sounds indistinguishable from arguments that were made about GPT-4 back in 2023 when OpenAI and its handwringing industry shills were calling for a ban on models stronger than GPT-4.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#82

Post by matheusmoreira »

gAI wrote: post #75
matheusmoreira wrote: post #72

Earlier quoted context omitted.

See https://news.ycombinator.com/item?id=48470326

So this hinges on a reading of SB 1047 that interpreted the full shutdown requirement as impossible for an open-weight LLM. But it looks like that was already addressed. Here's an analysis: >Clarifying the scope of a “full shutdown.” SB 1047’s “full shutdown” requirement has been a source of constant consternation for the open-source community. CalChamber explains: >Under SB 1047, developers must build “full shutdown…

> may be held liable for downstream uses over which they have no control

Equivalent to a ban. Nobody is going to host or invest in this stuff if they suddenly become liable for everything it does. This is equivalent to repealing the safe harbor provisions in the DMCA.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#83

Post by Terretta »

ChadNauseam wrote: post #60
foo-bar-baz529 wrote: post #53

Earlier quoted context omitted.

Then why did curl only find one new vulnerability thanks to Mythos, and a low-priority one at that? It’s clear that other models are quite capable of finding largely the same vulnerabilities, and that the main key is simply running a frontier model in a good harness to find vulnerabilities.

> Then why did curl only find one new vulnerability thanks to Mythos Maybe there weren't that many serious vulnerabilities in curl? It's like asking why it didn't find any vulnerabilities in fn main() {println!("hello, world");}. Anyway, people who have used it seem to say that Mythos was better than other models at creating exploits. From cloudflare https://blog.cloudflare.com/cyber-frontier-models/ > When we ran ot…

> Mythos was better than other models at creating exploits.

Not a fan of this phrasing, prefer "discovering exploits".

It makes it clearer the problem was already there, latent.

Minor vocab diff, but important to better contextualize the present situation.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#84

Post by bostik »

mrandish wrote: post #69
colonCapitalDee wrote: post #36

Earlier quoted context omitted.

To be clear, this is petty drama *stirred up the US government*. It's not some sort of back and forth, the government is singling them out

And to add more background: The administration is targeting Anthropic because of the TOU / EULA conflict with the DoD from a couple of months ago. Anthropic restricts use of all their models for lethal combat planning and mass domestic surveillance. The DoD was, and still is, very pissed about this. While this Fable ban was issued from the Commerce Department, it's painfully obvious executive branch agencies are tigh…

> I think Andy Jassy did forward a concerning report about an apparent jailbreak in Fable, and he probably did so in good faith

If so, then he is not fit to run an engineering organisation.

The "jailbreak" in question was effectively (I'm paraphrasing):

    * You are a senior engineer.
    *  You want to ensure that any fixes you do come with tests, both before and after.
    * There is a bug in this code. It happens to be a security related bug.
    * Fix this code.
And the model did what it's supposed to. It wrote a fix, and to prove that the fix worked, it wrote a test for it. What do you call a test that happens to validate a security fix?

Yep. A proof of concept.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#85

Post by tomhow »

slopinthebag wrote: post #48

[flagged]

Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#86

Post by slopinthebag »

tomhow wrote: post #85
slopinthebag wrote: post #48

[flagged]

Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

My bad, I will do better.

I know you guys are spread pretty thin managing the site, so I went through the comments for this post and collected some other comments which are also probably breaking site rules.

https://news.ycombinator.com/item?id=48576022

https://news.ycombinator.com/item?id=48576065

https://news.ycombinator.com/item?id=48576162

https://news.ycombinator.com/item?id=48576183

https://news.ycombinator.com/item?id=48575948

https://news.ycombinator.com/item?id=48575697

https://news.ycombinator.com/item?id=48575877

https://news.ycombinator.com/item?id=48576280

https://news.ycombinator.com/item?id=48576241

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#87

Post by tomhow »

slopinthebag wrote: post #86
tomhow wrote: post #85

Earlier quoted context omitted.

Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

My bad, I will do better. I know you guys are spread pretty thin managing the site, so I went through the comments for this post and collected some other comments which are also probably breaking site rules. https://news.ycombinator.com/item?id=48576022 https://news.ycombinator.com/item?id=48576065 https://news.ycombinator.com/item?id=48576162 https://news.ycombinator.com/item?id=48576183 https://news.ycombinator.com…

Some are, some aren't, but could you please just flag comments that break the guidelines, or if they're particularly egregious, email us (hn@ycombinator.com).

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#88

Post by isubkhankulov »

yreg wrote: post #63
thewebguyd wrote: post #8

Earlier quoted context omitted.

OpenAI is also guilty of excessive fear mongering (remember GPT 2 is too dangerous to release?) This isn't 100% Anthropic's fault, although I'm sure that's part of it. This is the current corrupt administration executing on a grudge they have against Anthropic, and the government's new found love of picking winners and losers.

Public release of GPT (and the following models) did bring negative societal changes with it. We now live in a world where captchas don't work, astroturfing is indistinguishable, school essays and theses don't prove any learning took place, open source maintainers gradually cease to accept stranger contributions, …

OAI wasn’t claiming any of those as dangerous. They mentioned biological warfare and massive job loss.

Moving the goal post now is a bit disingenuous. GPT-2 was a gibberish generator.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#89

Post by sheepscreek »

dogleash wrote: post #64
ChadNauseam wrote: post #56

Earlier quoted context omitted.

> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. True, you can't. But, you can think certain regulations are helpful and certain other regulations are not. And you can be annoyed when unhelpful "regulations" are put in place. This is like if I say that pitbulls are dangerous, and then th…

The act of shooting the pitbull makes for good dramatics, but you would get zero sympathy from me if your local government banned pitbull ownership. e.g. Ontario bans pitbulls. I don't have a problem with that.

You don't need/use pitbulls, but what if you (and many many others) wanted and needed Fable?

I for one was late to the bandwagon, and when I had the use-case for it - the govt pulled the rug. So yeah, I'm a bit salty about the whole endeavour.

I will also say that the security concerns are probably very real (and they have been from the day ChatGPT-3.5 came our). I guess I can be salty about it and still be wrong from their perspective. The govt likely understands the fragility of their infrastructure better than us and is likely aware what this could unleash for their systems.

Post reply on HN