Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The classifiers Anthropic puts in front of Fable are too zealous

combine-lab.github.io

Post reply on HN
Print view View original

81–90 of 203 posts

Re: The classifiers Anthropic puts in front of Fable are too zealous

#81

Post by smcleod »

mft_ wrote: post #5

This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness. e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, de…

I've been working with it heavily since its first release. I use it for software architects, complex debugging and some development and I have not had it refuse or downgrade even once.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#82

Post by llm_nerd »

charcircuit wrote: post #14
junebash wrote: post #11

This honestly just reads as “this model failed exactly where the company said it would but I’m very special and deserve special treatment rather than the same overactive guardrails I and everyone else were told we would get.”

When did Anthropic say you couldn't use it for math?

If you feed their "pure math" question to Fable, in its reasoning it rightly determines that it is the sort of thing you find in phylogenetics / algebraic-combinatorics complexity papers. That is what triggers the classifier.

Anthropic is 100% to blame for fear-mongering, but they said it would be blocked from any biology questions -- even high school level -- and they meant it. If the classifier sees anything related to biology, even in its own reasoning about the question, it blocks it.

Saying it's therefore not useful generally is of course ridiculous. Is it annoying? Of course it is.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#83

Post by SubiculumCode »

markbao wrote: post #43

[flagged]

the title was flamebait, but its true for him...and for me. I do neuroimaging. It won't answer any question / coding task having to do with data analysis / statistical analysis. etc. It IS useless, for me.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#84

Post by tsunamifury »

tracerbulletx wrote: post #34
prymitive wrote: post #19

Earlier quoted context omitted.

I think the problem here is that LLMs aren’t really “intelligence models” but more like “knowledge models”. LLMs don’t “think”, they just use a clever trick to make it seem like they do. I might not understand a lot about current state of AI, but that’s what they seem to be. Give it information and ask to organise it and make links, and they’ll do it, but that’s it, they don’t continually try to get out of the knowle…

When you watch it solve complex problems and use the browser and do internet searches, and use the entire surface area of the console tools on a linux box every day the idea that there are no major Homomorphisms with biological thinking is just completely out of the question. I also never understand what the difference between a thinking trick, and "real" thinking is supposed to be.

I used to agree with you but overtime I’ve changed my mind.

For reference I created predictive linguistics at Google in the first products and this is a many order scale up of that, with new complexities of course.

The best analogy I can give you is that it is a really advanced synthesis machine, which looks like human thought but is more of a hyper advance “replay” of human thought in various contexts.

Where you begin to see it fail is when it has no awareness of false paths in long walks, less awareness of getting stuck, and of course no unprompted intrinsic motivation.

This of course calls into question human thought being more than the rational mind but a mix of whole body input, biological needs, complex chemical behaviors and stored DNA information playing out after millions of years of evolution to build many different cooperating models of our “consciousnesses” and biological motivations .

Where as an LLM is more of an advance replay of the stored knowledge we bothered to record, synthesized into an execution in code.

It can do the things you’ve quoted because it has many recorded observations of those

Stick it in a robot and see how “smart” it is as everyday tasks. Give it a self oriented task and watch it mirror itself into oblivion.

It’s an advance thought extension system based on our history.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#85

Post by ACCount37 »

cubefox wrote: post #78
mft_ wrote: post #5

This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness. e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, de…

My guess is the classifier guardrails were made significantly stricter to convince the US government to reverse the ban.

Nah, the classifier was utterly asinine ON release. I'm not sure they could have made it worse if they tried.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#86

Post by thierrydamiba »

smcleod wrote: post #81
mft_ wrote: post #5

This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness. e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, de…

I've been working with it heavily since its first release. I use it for software architects, complex debugging and some development and I have not had it refuse or downgrade even once.

Have they ever talked about what goes into the classifier? I wonder how much your past chats impact it.

For example if it knows you do X at Y company is it more or less strict?

Re: The classifiers Anthropic puts in front of Fable are too zealous

#87

Post by robot_jesus »

smcleod wrote: post #81
mft_ wrote: post #5

This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness. e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, de…

I've been working with it heavily since its first release. I use it for software architects, complex debugging and some development and I have not had it refuse or downgrade even once.

Same. I really am curious if either 1) I am using it in genuinely different ways or 2) these people are being willfully disingenuous.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#88

Post by thx67 »

_davide_ wrote: post #80
vardalab wrote: post #18

Fable was refusing to patch vllm for me when trying to get mtp to work on r9700 gpus. Kept on bumping down to opus. Tried to really sanitize my prompts and everything but it seemed intrinsically prohibited from doing this sort of work. I guess it’s useful for making inane one shot games and websites, lol.

same experience here, as soon as it touched any gpu code it stopped working

I had it refuse OpenCL code that had nothing to do with anything, just slightly advanced cellular automata.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#89

Post by alwa »

FireBeyond wrote: post #61
markbao wrote: post #43

[flagged]

> the very thing Anthropic says it's not good for Where? Certainly not in its announcement, for one: https://platform.claude.com/docs/en/about-claude/models/intr... No "don't use this for X".

I thought it was the very first line of the product announcement, where they defined what it was they were calling "Fable" as opposed to "Mythos" in the first place:

https://www.anthropic.com/news/claude-fable-5-mythos-5

> Today we’re launching Claude Fable 5: a Mythos-class model that we’ve made safe for general use.

It then goes on to a lengthy and detailed section outlining the safety considerations:

https://www.anthropic.com/news/claude-fable-5-mythos-5#:~:te...

Re: The classifiers Anthropic puts in front of Fable are too zealous

#90

Post by kajman »

XorNot wrote: post #46
rcoveson wrote: post #33

Earlier quoted context omitted.

And biology is by far the classifier's least favorite topic. It's not even close. I've had it downgrade to Opus for the following questions: "How confident are we that English and American Eels both spawn in the Sargasso Sea?" "Come up with five Zoology questions of increasing difficulty for a trivia game." "What's your favorite sarcopterygian?" My wife has some zoology-related preferences in her user instructions, a…

It feels like the longtermist believers got involved in this (those are the people obsessed with garage-engineered designer viruses who have a very tenuous grasp on how biology research actually works).

I assumed they just wanted to cultivate FOMO to sell an even more expensive version to researchers later on.
Post reply on HN