Live data from Hacker News

Anthropic apologizes for invisible Claude Fable guardrails

theverge.com

341–350 of 489 posts

Re: Anthropic apologizes for invisible Claude Fable guardrails

#341
post #335

Can you imagine if Excel just quietly adjusted formulas in the background, and you didn't know the numbers weren't right? Or if Excel just said, Sorry, you can't use that formula with this formula? Or with these types of numbers, or this shape of data, etc?

you invest billions of dollars many months of work to just everyone distill your model?

You invest billions of dollars in hosting and benefit from hundreds of millions of man hours of human output, just so everyone trains on "your" data?

Re: Anthropic apologizes for invisible Claude Fable guardrails

#342
How did people read this action in such a weird ultra me centric way? Distillation is such a big problem that distill attempts make up a significant share of their revenue (!).

A distilled model can be used to rob your grandma in a highly effective way. This isn't about placing a few business-logic rules in JS + CSS on your website anymore. Wake up.

A distilled model with an easy jailbreak can be used to coordinate terrorist attacks or hostile state operations... think Russia, North Korea, and the like.

Re: Anthropic apologizes for invisible Claude Fable guardrails

#343
post #329

Can you imagine if Excel just quietly adjusted formulas in the background, and you didn't know the numbers weren't right? Or if Excel just said, Sorry, you can't use that formula with this formula? Or with these types of numbers, or this shape of data, etc?

They implemented both those things, but only apologized for the first. They’re doubling down on the second. My limited experience with fable over the last few days suggests (1) I can’t see any improvement in output, and (2) it is useless for writing secure software because it constantly hits safety walls if you ask it to close security holes. I’m definitely shopping around for other LLM providers next week, and testi…

With 128 GB strix halo, you can't do as big of a model as you would think. You can do larger than having a single graphics card, of course, but that 128 gigs cannot all be dedicated to the model. Remember, the context alone is usually larger than the model itself. I got an EVO X2, and I don't regret it, but by my current calculations, it will take 8 years to recoup the cost, as opposed to just using equivalent, paid commercial options.

Re: Anthropic apologizes for invisible Claude Fable guardrails

#344
post #342

How did people read this action in such a weird ultra me centric way? Distillation is such a big problem that distill attempts make up a significant share of their revenue (!). A distilled model can be used to rob your grandma in a highly effective way. This isn't about placing a few business-logic rules in JS + CSS on your website anymore. Wake up. A distilled model with an easy jailbreak can be used to coordinate t…

Imagine if your IDE started injecting bugs into your project just because your code looked like it implemented a competing IDE.

Re: Anthropic apologizes for invisible Claude Fable guardrails

#345

I develop some deep learning models. They don't compete with Anthropic, nor are they language models. They mostly enable mathematical optimization systems to approximate actual the actual physics of radio propagation models with a fraction of the latency/compute of a high resolution simulator. Technically that should be safe for me to use with Claude Code, but how the fuck am I supposed to know? You're degrading/malw…

Same here, I fine tune LLMs for specific use cases. How can I trust Anthropic models not to introduce bugs to preserve their moat?

Re: Anthropic apologizes for invisible Claude Fable guardrails

#346
post #342

How did people read this action in such a weird ultra me centric way? Distillation is such a big problem that distill attempts make up a significant share of their revenue (!). A distilled model can be used to rob your grandma in a highly effective way. This isn't about placing a few business-logic rules in JS + CSS on your website anymore. Wake up. A distilled model with an easy jailbreak can be used to coordinate t…

Imagine if your IDE started injecting bugs into your project just because your code looked like it implemented a competing IDE.

how is that related. It downgrade it to opus 4.8 #2 most capable model after claude 5. for a vast majority of topics it will not downgrade. I've been using it for 2 days to talk about architecture etc. and it was absolutely great with no downgrades.

Re: Anthropic apologizes for invisible Claude Fable guardrails

#347
post #335

Earlier quoted context omitted.

you invest billions of dollars many months of work to just everyone distill your model?

That might be an indication that the business is not sustainable because there is not any technical or practical differentiator besides scale. Harming your customers to maintain that differentiation isn't sustainable either.

any intellectual labor is not sustainable, if anyone can copy your data. why have microsoft, i you can just copy windows and run it?

Re: Anthropic apologizes for invisible Claude Fable guardrails

#348
post #347

Earlier quoted context omitted.

That might be an indication that the business is not sustainable because there is not any technical or practical differentiator besides scale. Harming your customers to maintain that differentiation isn't sustainable either.

any intellectual labor is not sustainable, if anyone can copy your data. why have microsoft, i you can just copy windows and run it?

Have you copied Windows and tried to run it? I would love to see the plain text source code that you claim to have. We all would.

Re: Anthropic apologizes for invisible Claude Fable guardrails

#349
post #329

Earlier quoted context omitted.

They implemented both those things, but only apologized for the first. They’re doubling down on the second. My limited experience with fable over the last few days suggests (1) I can’t see any improvement in output, and (2) it is useless for writing secure software because it constantly hits safety walls if you ask it to close security holes. I’m definitely shopping around for other LLM providers next week, and testi…

With 128 GB strix halo, you can't do as big of a model as you would think. You can do larger than having a single graphics card, of course, but that 128 gigs cannot all be dedicated to the model. Remember, the context alone is usually larger than the model itself. I got an EVO X2, and I don't regret it, but by my current calculations, it will take 8 years to recoup the cost, as opposed to just using equivalent, paid…

A key consideration in favor of running your local LLM despite all the trouble: The commercial serving endpoint may not exist tomorrow, or at least not at the same price.
Post reply on HN