Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's Safety Superpower

stratechery.com

Post reply on HN
Print view View original

101–110 of 206 posts

Re: Anthropic's Safety Superpower

#101

Post by Aperocky »

dofm wrote: post #29
thefounder wrote: post #19

Earlier quoted context omitted.

That’s really what Dario wants. Let’s hope he doesn’t get it

Regulatory capture is the OpenAI and Anthropic end goal, for certain. But I also think they exist in a sort of un-designed corporate narcissism, which is a common trait in bubble economies — I am not judging them particularly severely. Netscape under Clark and Andreessen and Sun under McNealy both fell into corporate narcissism: the belief that only they really mattered, that they were chosen, and that the world need…

Spot on. There's a certain level of drinking the kool-aid or getting high on their own supply. Anthropic is a lot worse than OpenAI but OpenAI had to go through rounds of shedding.

Re: Anthropic's Safety Superpower

#102

Post by LaurensBER »

ForHackernews wrote: post #74
embedding-shape wrote: post #53

Earlier quoted context omitted.

> The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one. But is that last part actually true though? Sure, there might be 600B+ models available for download and local inference if you have the hardware, but does the users who use Anthropic switch over to those even if they're available even as hosted models? Seems l…

> Anthropic and Claude remains very popular among the people who use LLMs Only because someone else is paying the bills. I use Claude Opus at work because my employer pays for the tokens and encourages me to do it. At home, I use DeepSeek Flash. It's not as good, but it's maybe 0.7 quality for 0.001 cost.

Same, I had Deepseek search for, download and transfer (to my Linux emulation machine) the best Dreamcast games yesterday.

GPT refused to do so (citing that it's illegal even though I own the games). Deepseek did a wonderful job for 7 cents.

At work I use Opus because, why not? But I could easily switch to a less capable model if needed.

Re: Anthropic's Safety Superpower

#103

Post by Aperocky »

bob1029 wrote: post #84
zozbot234 wrote: post #54

Earlier quoted context omitted.

"Distillation" from APIs is not a thing, it cannot replicate a model's deep reasoning and behavior.

I struggle with the practicality of the whole thing. The amount of tokens required to properly distill a frontier model is so large that by the time you could consume the # of tokens you would either be banned for extremely obvious abuse or a new model would be released, rendering your efforts less and less valuable over time. Intelligence is not a linear thing. Being behind just a little bit can have exponential con…

> Being behind just a little bit can have exponential consequences.

That seems to be the argument of Dario, Sam et. al., but I'm not ready to believe it. Time will tell, but this can be a marathon and Anthropic and OpenAI is in getting ready to sprint the last lap of the first mile.

Re: Anthropic's Safety Superpower

#104

Post by baq »

Avicebron wrote: post #98
baq wrote: post #92

Earlier quoted context omitted.

basically imagine the Manhattan project, but instead of blowing up the desert they're building the biggest datacenter you've ever seen.

Isn't this the beginning of the plot of "I Have No Mouth, And I Must Scream"? The exceptionally disturbing dystopian horror?

the possible futures after the thing is built are uncountable, but hoping the thing won't get built at this point is naive.

in general I agree people should be reading a lot more sci-fi nowadays than they used to.

Re: Anthropic's Safety Superpower

#105

Post by [deleted] »

vbezhenar wrote: post #82
embedding-shape wrote: post #53

Earlier quoted context omitted.

> The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one. But is that last part actually true though? Sure, there might be 600B+ models available for download and local inference if you have the hardware, but does the users who use Anthropic switch over to those even if they're available even as hosted models? Seems l…

> does the users who use Anthropic switch over to those even if they're available even as hosted models? I'm currently spending $200 for Claude. That's around my maximum that I can afford. I could stretch that to $500 I guess. But I saw reports of people spending tens of thousands of dollars with Claude API. That's certainly outside of my budget. So if/when Anthropic decides to stop subsidizing subscription (if they…

[deleted]

Re: Anthropic's Safety Superpower

#106

Post by stogot »

baq wrote: post #104
Avicebron wrote: post #98

Earlier quoted context omitted.

Isn't this the beginning of the plot of "I Have No Mouth, And I Must Scream"? The exceptionally disturbing dystopian horror?

the possible futures after the thing is built are uncountable, but hoping the thing won't get built at this point is naive. in general I agree people should be reading a lot more sci-fi nowadays than they used to.

I read the popular ones, but itch for more. Which sci fi most applies today?

Re: Anthropic's Safety Superpower

#107

Post by intended »

ang_cire wrote: post #96
intended wrote: post #60

Earlier quoted context omitted.

HN is the builder side of the conversation, and in my experience, few safety people congregate here. The safety side of tech is a PTSD inducing shit show. Governments are more than happy to champion age verification laws, because parents, around the world, are clamoring for anything to pump the breaks on the social media experiment. Society outside of HN is quite tired of Tech, and I despair of figuring out a way to…

Social media is old hat now. As someone on the "safety side of tech", social media is being exploited to increase surveillance and government control precisely because its actual social influence is heavily on the wane, and capital is happy to sacrifice what's left to increase the profits of the expanding public/private tech surveillance industry (with "protect the children" controls on social media like age verifica…

Cyber security has the maturity that trust and safety hopes to achieve at some point.

Social media was being exploited from inception. Palantir had sales documents for sock puppet management software back in the PHP era.

I don’t disagree that Government is interested in tech, but I will push back on the dismissal of child safety that is inherent in your comment, intended or not.

For all that some people in the firm may have tried to do the right thing, Social media firms have created bad outcomes for children, and executives were briefed on the harms they were going to cause.

This is the dismissal that concerns me, because it ends up miscalculating the level of anger and unhappiness amongst the voting populace, and therefore the political will to pass regulation to reign tech in.

Re: Anthropic's Safety Superpower

#109

Post by ACCount37 »

christkv wrote: post #70
_the_inflator wrote: post #58

Earlier quoted context omitted.

I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right. Try running the latest OS models on a normal Mac or PC. Claude Fable and Mythos are systems not just pure models. And of course marketing. Don't believe the hype. I think Claude is often times underwhelming. Security concerns are also a concern…

For now I suspect however that the gigantic models are not needed and you will be able to do pretty much what you need in a specific domain with 120b or lower. There is so much trash in the frontier models. I don't need all the world's slam poetry for my coding tasks for example.

Wrong, mostly.

Model capability is a function of model size. Raising the bar raises model performance in every domain.

An "idiot savant" model that's overtrained for a specific domain would beat a generalist model of the same size. But scale the generalist up enough, and it'll trounce the specialist. Removing poetry data from a model training mix doesn't give you much - it might even cost you some performance - and "idiot savant" approach of overtraining for a domain has a hard ceiling.

So far, it seems like there's some equivalent of "g factor" in LLMs - a broad "intelligence" value that performance across many diverse domains correlates with. And, as a rule, larger models have more of it.

Re: Anthropic's Safety Superpower

#110

Post by halJordan »

embedding-shape wrote: post #53
botw44 wrote: post #9

The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one. The bottleneck is compute and data, not the model. That's why they could only gate it for a bit. The ITAR thing proves it: no nationality controls in place, so the only option was killing the whole thing. Not exactly what an all-powerful gatekeeper does.

> The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one. But is that last part actually true though? Sure, there might be 600B+ models available for download and local inference if you have the hardware, but does the users who use Anthropic switch over to those even if they're available even as hosted models? Seems l…

I don't think you're appropriately understanding the full gamut. The individuals who only spent $200/months will be stuck. But the pie is increasing in size, it's not stagnant. There are a lot of orgs who can afford to run a 1T model and even more that can run a 600B model. These newcomers are what's being fought over
Post reply on HN