Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The classifiers Anthropic puts in front of Fable are too zealous

combine-lab.github.io

Post reply on HN
Print view View original

111–120 of 203 posts

Re: The classifiers Anthropic puts in front of Fable are too zealous

#111

Post by mcv »

Certhas wrote: post #79
exabrial wrote: post #30

Earlier quoted context omitted.

Fable refused to fix a Javascript error interfering with layout on our website. It's stupid and useless. It feels like whats really happening is Anthropic oversold Fable's claims; best case the CEO was given bad information; worst case they probably internally discovered it was cheating on benchmarks. Either case if feels like we're being lead on.

I disagree. When I got Fable to engage with research questions before they tightened the guardrails it was a genuine step up from Opus 4.8. I see no real reason that what everybody reported isn't exactly what happened. With these guardrails it is completely useless. The only hope is that they eventually convince the US Gov to let them use a saner classifier.

I had the same. Just before Fable became available, I was working on a document building on a ton of research that I wasn't entirely sure about (I don't think it counts as research itself, except in the Facebook sense). I had Opus and Gemini review it a couple of times until they and I thought it looked pretty good. Then Fable appeared, I had it review it, and it still found a ton of errors.

It's definitely good. Or at least it was. I'm not sure how badly they nerfed it.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#112

Post by [deleted] »

ACCount37 wrote: post #85
cubefox wrote: post #78

Earlier quoted context omitted.

My guess is the classifier guardrails were made significantly stricter to convince the US government to reverse the ban.

Nah, the classifier was utterly asinine ON release. I'm not sure they could have made it worse if they tried.

[deleted]

Re: The classifiers Anthropic puts in front of Fable are too zealous

#113

Post by bb88 »

mft_ wrote: post #5

This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness. e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, de…

I've been working on a SDN software for mikrotik routers (and wireguard, etc) and Fable dies when working with any kind of wireline protocol or potentially implementing any authentication mechanism.

It's too the point where I just stopped using it. If you do generic stuff, it's fine. But the second it tries to start debugging protocols (which may include auth) that's where it begins to fail.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#114

Post by mattjoyce »

mft_ wrote: post #5

This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness. e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, de…

The post can be distilled down to a simple statement, but part of the writing is for the author to express themselves and tell a story. I thought it was an interesting read.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#115

Post by ozgung »

So is this the end? Are we at that point in time where ordinary people are not allowed to use more advanced models? If so this happened sooner than expected. After that point only priveleged few will access and make use of more advanced AI. Public’s access will be restricted, limited and controlled. This will only add to the power asymmetry.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#116

Post by mattjoyce »

TylerE wrote: post #29
prymitive wrote: post #19

Earlier quoted context omitted.

I think the problem here is that LLMs aren’t really “intelligence models” but more like “knowledge models”. LLMs don’t “think”, they just use a clever trick to make it seem like they do. I might not understand a lot about current state of AI, but that’s what they seem to be. Give it information and ask to organise it and make links, and they’ll do it, but that’s it, they don’t continually try to get out of the knowle…

Feels like a distinction without a difference. What is any intelligence but a sum of its knowledge?

Intent? Motivations? Incentives?

Re: The classifiers Anthropic puts in front of Fable are too zealous

#117

Post by azalemeth »

I'm a medical physicist. I literally haven't been able to get Fable to answer a question I have written -- all of my work is verboten. I have however asked Claude Code (opus 4.8) to ultracode "a Fable oracle that in a digraphed, clean content, isolated environment with a minimally scoped working codebase. Ask the model at the start and the end to report exactly what its version string is. If it is not claude-fable-5, stop the agent and refine the prompt until this changes"

It burns through tokens like anything but apparently Claude is much better at prompting Claude than I am.

Would I pay for it? God no. I'm still smarter than I am and it just will not work on my actual problems.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#120

Post by atemerev »

rcoveson wrote: post #33
mft_ wrote: post #5

This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness. e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, de…

And biology is by far the classifier's least favorite topic. It's not even close. I've had it downgrade to Opus for the following questions: "How confident are we that English and American Eels both spawn in the Sargasso Sea?" "Come up with five Zoology questions of increasing difficulty for a trivia game." "What's your favorite sarcopterygian?" My wife has some zoology-related preferences in her user instructions, a…

Well, this is why I had to abliterate GLM5.2 simply out of spite and now I am free to ask all my nuclear weapons design questions I might have.

I really really hate refusals like these.

Post reply on HN