[flagged]
The classifiers Anthropic puts in front of Fable are too zealous
161–170 of 203 posts
Re: The classifiers Anthropic puts in front of Fable are too zealous
#162Fable was refusing to patch vllm for me when trying to get mtp to work on r9700 gpus. Kept on bumping down to opus. Tried to really sanitize my prompts and everything but it seemed intrinsically prohibited from doing this sort of work. I guess it’s useful for making inane one shot games and websites, lol.
> Tried to really sanitize my prompts
And in doing so, you probably got your account and prompt flagged for 'attempted jailbreaking' (apparently, such scores are remembered for up to 7 years).
Re: The classifiers Anthropic puts in front of Fable are too zealous
#163Earlier quoted context omitted.
Yeah i'm wondering how much of a role that plays in this as well. On the one hand I could believe it's something more benign, or the usual misunderstood fear mongering making it to some political level (well make sure those users can't get online anonymously! being our current craze). That said, chemistry and to some level physics have been the major domain of limited knowledge (chemistry because the average person c…
Nobody has tried to limit knowledge of chemistry or physics unless it was directly about doing something illegal, to the point of basically being a detailed recipe. Usually not even then. And when they have tried they've had basically zero success. The ability for a handful of companies, simultaneously very powerful and easily susceptible to pressure from other powerful actors, to do the same sort of thing with the n…
I agree, and think the effects on learning should be doubly emphasized. One can lock down everything and everyone to the highest degree possible, think of every possible edge case, set controls 2, 3, 4, 10 steps away from them, but not only is this not beneficial to society overall due to how it hurts adjacent information, it's not even beneficial to the goal in question, since it creates a brittle situation with locks that can't be changed or updated in a world which is always changing and always updating.
Re: The classifiers Anthropic puts in front of Fable are too zealous
#164The most basic machine learning-related query gets flagged for me. For example:
In flax nnx, what's the idiomatic way to store state on a Module. For example, if I'm handling the carry manually for an nnx.RNN.
How do I restore one of the orbax checkpoints into NNX from this script?
It's a shame - I like Fable for writing tasks over ChatGPT and I do believe Anthropic is a more ethical outfit than OpenAI. But with the safeguards (and Fable access expiring in a few days) there's no reason to pay for draconian guardrails and harsh rate limits.
Re: The classifiers Anthropic puts in front of Fable are too zealous
#165This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness. e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, de…
That's extremely related to biology.
Re: The classifiers Anthropic puts in front of Fable are too zealous
#166I must apologize to my devoted followers on this program, but if the author can't bother to read the giant yellow warning at the top of the screen, I can't be bothered to finish the essay!
Re: The classifiers Anthropic puts in front of Fable are too zealous
#167Earlier quoted context omitted.
The author is working on an opensource C++ codebase and not on biology tasks. The work is around tooling. It's like saying well a scalpel is used for medical reasons, sure. But manufacturing scalpels is metalworking, not medicine.
I think it's accurate to characterize the project as bio-related work: https://github.com/COMBINE-lab/salmon > salmon is a wicked-fast program for highly-accurate, transcript-level quantification from RNA-seq data. It pairs a fast mapping stage — selective alignment, or alignment-free sketch mode (--sketch) — with a massively-parallel statistical model (EM/VBEM over equivalence classes) to estimate transcript abundan…
Got it, so all those advances in medicine we were promised in exchange for higher electricity costs, global warming, and other pitfalls of AI were bunk?
Re: The classifiers Anthropic puts in front of Fable are too zealous
#168This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness. e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, de…
I've been working on a SDN software for mikrotik routers (and wireguard, etc) and Fable dies when working with any kind of wireline protocol or potentially implementing any authentication mechanism. It's too the point where I just stopped using it. If you do generic stuff, it's fine. But the second it tries to start debugging protocols (which may include auth) that's where it begins to fail.
I can’t even use it to fix the bugs Opus introduced. I’ve considered ripping out auth until fable is behind the paywall. I’ve been very careful in my queries and broken everything down to careful segments. Even the memory can get security verbs poisoning further requests.
Re: The classifiers Anthropic puts in front of Fable are too zealous
#169Try asking it to do a simple code review. Literally no prompt other than their own code-review skill. It triggers a safety flag almost 80-90% of the time.
Re: The classifiers Anthropic puts in front of Fable are too zealous
#170Earlier quoted context omitted.
My experience, too. I work on nothing in any way related to cybersecurity or biology. I asked it a few purely mathematical questions, it refused immediately. Before the export embargo I did get it to look at some hairy problems and the output was genuinely useful...
Were your question in math areas related to ML? They also restrict model development and research pretty heavily.
Getting it to work on anything involving ai coding agents is a pita. Ironically. Even if it’s just at the application level.