Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The classifiers Anthropic puts in front of Fable are too zealous

combine-lab.github.io

Post reply on HN
Print view View original

91–100 of 203 posts

Re: The classifiers Anthropic puts in front of Fable are too zealous

#91

Post by Certhas »

scrumbledober wrote: post #49
markbao wrote: post #43

[flagged]

there's plenty of uses for models not doing what they were made to do, but this is even worse. It's people trying to get the model to do what it was made specifically not to do!

You are mischaracterizing what the post is reporting entirely. Porting an open source tool used in bioscience to rust is a software engineering task. But it is somewhat understandable that it gets stuck in the overly broad safety margin.

But I do research on stuff that is entirely unrelated to bio or cybersecurity, and the model is simply not taking any of my research-level prompts. This is fairly abstract mathematical stuff. All of this, including all the examples in the posted article, are far from "trying to get the model to do what it was made not to do".

Re: The classifiers Anthropic puts in front of Fable are too zealous

#92

Post by Telemakhos »

slowin wrote: post #6

Do we think that someone at Anthropic, OpenAI, the government... has access to SOTA models without censorship? "How do I build an effective weapon?", "How do I effectively control the masses?"... It's very concerning that we get the nerfed models but you know that somewhere, people with a lot of resources have access to the raw, uncensored, probably more powerful models. The sprint toward AGI looks even more dangerou…

"How do I build an effective weapon?" and "How do I effectively control the masses?" were research projects for the US government before you were born. One gave the world the Manhattan project, to name only one example, and the other MKULTRA. The government and cooperating companies had knowledge in both fields beyond the state of the art publicly available and continue to hold that edge over the public and foreign adversaries today. There is precious little new about the government having an uncensored model while you get the nerfed version.

A useful comparison might be made with the realm of firearms: civilians need to jump through hoops to own a fully-automatic weapon and can run afoul of the law simply by drilling a third hole near two others in a hunk of metal, yet the better trained among the government's soldiery can operate fully automatic weapons. You get the nerfed version, and the BATFE will have problems if you try to circumvent that restriction. I wonder, though, how many people advocating for popular access to uncensored AI models also advocate for an unrestricted (not infringed) right to bear arms or an unrestricted right to freedom of speech.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#93

Post by epolanski »

I goddamn hate fable for anything but vibecoding.

It's generally a major downgrade in acting like an assistant.

I don't know what's wrong but it is just bad at multi turn discourse even on a limited amount of content with no MCP or bash calls of any sort.

The thing that makes me mad is how stubbornly confident it is even whets wrong.

I have to tell it many times to actually re read the conversation as it even insists I said something else.

It's like it had a scratchpad where it has some summarized bullet points which it fills of made up content.

I'm so confused. On one side I like to connect it to honeycomb/otel logs and I can see it figures out difficult bugs in the code better than other models.

On some others I feel I'm assisting at a continuos disaster and consistent degradation since Opus 4.6, it's a tragedy.

I'm more and more the assistant to a capable, yet confidently stubborn and wrong LLM.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#94

Post by fer »

wolttam wrote: post #62
vardalab wrote: post #18

Fable was refusing to patch vllm for me when trying to get mtp to work on r9700 gpus. Kept on bumping down to opus. Tried to really sanitize my prompts and everything but it seemed intrinsically prohibited from doing this sort of work. I guess it’s useful for making inane one shot games and websites, lol.

I was recently using self-hosted DeepSeek V4 Flash to poke around the DSpark implementation in vLLM (well outside of my domain) I did wonder if I was doing anything Fable would have flagged - sounds like yes.

Fable refuses to touch anything in my side project because it uses libtorch. It will bounce even for parts that have nothing to do with it.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#95

Post by DannyBee »

the ancestry predicate at the beginning of the formal problem statement here is dominance, at least as applied to their rooted trees.

Because it is a rooted tree, only DFS intervals are required to determine ancestry.

You can detect whether a new blocking loop is going to be formed through online dominator maintenance/online cycle detection, etc, during optimization, rather than use a heuristic, if you wanted to.

Not sure it's practically faster, but that's at least the graph-theoretic answer.

In practice, outside of the suggested heuristic, I have to imagine you'd normally throw branch and bound at this, using some lazy-cut for the blocking loops (IE you can keep any of these edges but not all of them) and let it go to town.

The paper (at least, this paper) doesn't compare that to what they did, and i'd be shocked if someone hasn't tried this before, so not sure it's useful.

I'll also say you can get existing AI models to tell you the above, but you have to push them a bit most of the time step by step. Just handing them the whole overall problem, as described, and saying "what are the graph theoretical problems related to this" it sort of gets lost.

Probably because the LLM isn't doing a good job of predicting graph-theoretic words when the language is not graph theoretic, but if you translate it into a graph theoretic language piece by piece, and ask it about that, the prediction becomes better :)

Re: The classifiers Anthropic puts in front of Fable are too zealous

#96

Post by prox »

matthewdgreen wrote: post #72
scrumbledober wrote: post #49

Earlier quoted context omitted.

there's plenty of uses for models not doing what they were made to do, but this is even worse. It's people trying to get the model to do what it was made specifically not to do!

We don't need top-end frontier models to write simple applications. Opus works very well for that and it's cheaper. We need them to write things that are at the frontier.

We need? Do we?

Re: The classifiers Anthropic puts in front of Fable are too zealous

#98

Post by girvo »

Eji1700 wrote: post #64
XorNot wrote: post #46

Earlier quoted context omitted.

It feels like the longtermist believers got involved in this (those are the people obsessed with garage-engineered designer viruses who have a very tenuous grasp on how biology research actually works).

Yeah i'm wondering how much of a role that plays in this as well. On the one hand I could believe it's something more benign, or the usual misunderstood fear mongering making it to some political level (well make sure those users can't get online anonymously! being our current craze). That said, chemistry and to some level physics have been the major domain of limited knowledge (chemistry because the average person c…

The thing is the data isn’t limited, and supply side constraints already solve this problem. I come from a BSc Chemistry background, and they don’t hide how organic chemistry and illegal drug synthesis are intertwined, it’s open information

But where I live the glassware and precursors will get you a very angry knock on the door.

Re: The classifiers Anthropic puts in front of Fable are too zealous

#100

Post by peyton »

XorNot wrote: post #46
rcoveson wrote: post #33

Earlier quoted context omitted.

And biology is by far the classifier's least favorite topic. It's not even close. I've had it downgrade to Opus for the following questions: "How confident are we that English and American Eels both spawn in the Sargasso Sea?" "Come up with five Zoology questions of increasing difficulty for a trivia game." "What's your favorite sarcopterygian?" My wife has some zoology-related preferences in her user instructions, a…

It feels like the longtermist believers got involved in this (those are the people obsessed with garage-engineered designer viruses who have a very tenuous grasp on how biology research actually works).

I but skimmed the model card on release, but my impression was that there may be an incentive for this expert panel to exaggerate as a form of job security. A lot of the challenges seemed to be of the form “would this allow somebody who isn’t me to do what I do professionally?”
Post reply on HN