Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The hacker sent by Anthropic to calm the government's nerves about AI safety

wsj.com

Post reply on HN
Print view View original

51–60 of 131 posts

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#51

Post by bloppe »

micromacrofoot wrote: post #37
cmiles8 wrote: post #9

The AI labs look rather naive here. You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. Their new argument now seems be that this was marketing hype/fluff that backfired, in a pretty obvious and predicable way, and now they’re trying to reset the conversation.

I suspect they're taking this as a win either way, because they're still plastering "Fable 5 unavailable" on their product and using it as an opportunity to keep themselves in the spotlight as they head to IPO. There's really not even a ban here, they could slot in Fable under the Opus label and no one would really be able to tell. It's all part of the same show to pump up valuation.

I bet they will do a touch of RLHF and re-naming the moment OAI releases a comparable model. Otherwise, sure, they can just bask in the drama for a bit.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#52

Post by drtz »

cmiles8 wrote: post #9

The AI labs look rather naive here. You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. Their new argument now seems be that this was marketing hype/fluff that backfired, in a pretty obvious and predicable way, and now they’re trying to reset the conversation.

> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it.

It's entirely possible that models could be "dangerous" to fully release to the general public without guardrails and at the same time the government majorly overreacted in this case.

Releasing Mythos to selected researchers and companies at least gives those researchers a head start at addressing vulnerabilities before the model hits mainstream.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#53

Post by foo-bar-baz529 »

drtz wrote: post #52
cmiles8 wrote: post #9

The AI labs look rather naive here. You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. Their new argument now seems be that this was marketing hype/fluff that backfired, in a pretty obvious and predicable way, and now they’re trying to reset the conversation.

> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. It's entirely possible that models could be "dangerous" to fully release to the general public without guardrails and at the same time the government majorly overreacted in this case. Releasing Mythos to selected researchers and companies…

Then why did curl only find one new vulnerability thanks to Mythos, and a low-priority one at that? It’s clear that other models are quite capable of finding largely the same vulnerabilities, and that the main key is simply running a frontier model in a good harness to find vulnerabilities.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#54

Post by binary132 »

bellowsgulch wrote: post #26
cmiles8 wrote: post #9

The AI labs look rather naive here. You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. Their new argument now seems be that this was marketing hype/fluff that backfired, in a pretty obvious and predicable way, and now they’re trying to reset the conversation.

[flagged]

that seems like possibly the most unlikely outcome

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#55

Post by AbrahamParangi »

Simon321 wrote: post #5

These are the consequences of fear mongering as hard as they did. You reap what you sow. Now they need to convince the government that they didn't mean anything of the previous things they claimed.

Nah, that's ridiculous. This admin is corrupt and idiotic and it's silly to pretend that Anthropic's actions matter except in so much as they didn't bribe the president like OpenAI did.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#56

Post by ChadNauseam »

cmiles8 wrote: post #9

The AI labs look rather naive here. You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. Their new argument now seems be that this was marketing hype/fluff that backfired, in a pretty obvious and predicable way, and now they’re trying to reset the conversation.

> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it.

True, you can't. But, you can think certain regulations are helpful and certain other regulations are not. And you can be annoyed when unhelpful "regulations" are put in place.

This is like if I say that pitbulls are dangerous, and then the government comes and shoots my pitbull, who I've spent a lot of effort training to not be dangerous. Then you say "well you said pitbulls were dangerous, so you can't really complain." Well, I can complain because If you took me seriously, you wouldn't have responded by shooting only my pitbull!

Think of what incentives this creates for other people. Do you think that OpenAI will be candid about the possible dangers of their technology now? They might not even release it now, seeing that Anthropic releasing their model was what got it export-controlled.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#57

Post by thewebguyd »

yeeetz wrote: post #29
tiahura wrote: post #10

They need to send lobbyists not hackers.

They needed to have administration insiders on their team months if not years ago, not just now OpenAI, Meta, SpaceX are savvy enough to play ball, but Anthropic's public posturing and government affairs has always seemed too aloof and intellectual

To be fair to Anthropic for a moment (not that they deserve it), but requiring administration insiders and the greasing of palms going on should not continue to be the normal expectations of how to do business in the USA. I'm on the side of any company that refuses to capitulate to this administration. Not saying Anthropic doesn't (because they do), but let's not pretend like the blatant corruption going on should be normalized. Every single citizen should be appalled at this behavior and blatant market manipulation.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#58

Post by matheusmoreira »

ChadNauseam wrote: post #49
matheusmoreira wrote: post #44

Earlier quoted context omitted.

We know, but it's still satisfying to see their fearmongering backfire on them.

If you "know" that it's "99% petty drama between the US government and Anthropic", then it's not really their fearmongering backfiring on them.

It absolutely is. They pretty much gave the government the perfect excuse to meddle in their operations.

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#59

Post by reje »

Bender wrote: post #50
cmiles8 wrote: post #9

The AI labs look rather naive here. You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. Their new argument now seems be that this was marketing hype/fluff that backfired, in a pretty obvious and predicable way, and now they’re trying to reset the conversation.

Also a good wake-up call for investors as these big players can be benched at any moment.

I can’t take anyone seriously who thought otherwise.

You think you can become more powerful so much so the govt questions its own power? Don’t be stupid. They will simply send in the army to first seize the assets and then nationalise.

It almost seems as if very few people actually understand how the world works. If the govt thinks this is the tech to end all future tech, you think future money flows for invesment matter? Hahaha. No

Re: The hacker sent by Anthropic to calm the government's nerves about AI safety

#60

Post by ChadNauseam »

foo-bar-baz529 wrote: post #53
drtz wrote: post #52

Earlier quoted context omitted.

> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. It's entirely possible that models could be "dangerous" to fully release to the general public without guardrails and at the same time the government majorly overreacted in this case. Releasing Mythos to selected researchers and companies…

Then why did curl only find one new vulnerability thanks to Mythos, and a low-priority one at that? It’s clear that other models are quite capable of finding largely the same vulnerabilities, and that the main key is simply running a frontier model in a good harness to find vulnerabilities.

> Then why did curl only find one new vulnerability thanks to Mythos

Maybe there weren't that many serious vulnerabilities in curl? It's like asking why it didn't find any vulnerabilities in fn main() {println!("hello, world");}.

Anyway, people who have used it seem to say that Mythos was better than other models at creating exploits. From cloudflare https://blog.cloudflare.com/cyber-frontier-models/

> When we ran other frontier models through the same harness, they found a fair number of the same underlying bugs, and in some cases they got further than we expected on the reasoning side too. Where they fell short was at the point of stitching the pieces together. A model would identify an interesting bug, write a thoughtful description of why it mattered, and then stop, leaving the actual chain unfinished and the question of exploitability open. What changed with Mythos Preview is that a model can now take those low-severity bugs (which would traditionally sit invisible in a backlog) and chain them into a single, more severe exploit.

Post reply on HN