Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If Claude Fable stops helping you, you'll never know

jonready.com

Post reply on HN
Print view View original

351–360 of 534 posts

Re: If Claude Fable stops helping you, you'll never know

#351

Post by SXX »

nsingh2 wrote: post #217
jsw97 wrote: post #184

Given the high rate of false positives people are reporting for the non-silent cybersecurity, biological, etc., safeguards, there is a strong likelihood that you will encounter silently nerfed behavior even if you are _not_ violating their TOS. Ultimately this will be evident in the way customers / external benchmarkers experience Fable. Hopefully competition will drive future models toward a lower false positive rat…

It's such an obviously bad policy, it's mind-boggling that they thought this was a good idea. It just breeds paranoia and mistrust, especially when people are already a bit paranoid about silent model quantification for cost cutting reasons.

Its not pranoia when entity you are dealing with cant be trusted and will do everything to abuse your trust.

Re: If Claude Fable stops helping you, you'll never know

#352

Post by DeathArrow »

theLiminator wrote: post #151
SwellJoe wrote: post #116

Earlier quoted context omitted.

I think we agree? What moat? You answered yourself: "capital intensive" But, history says the supercomputer of today will fit in your pocket in a few years. They've bought up all the RAM and GPUs, which pushes the capital requirements upward for everyone else. But, they can't corner the market forever, there are too many competing interests. AMD and Intel keep making new GPUs and APUs. The memory makers can't just se…

> But, history says the supercomputer of today will fit in your pocket in a few years. I don't think this will be true in the same time span anymore. Each miniaturization is costing more and more money. Perhaps they'll come up with exotic fundamental improvements, but I don't think the rate of improvement of compute/watt will match the previous decades.

>but I don't think the rate of improvement of compute/watt will match the previous decades.

Unless we invest heavily in research and find new way to do chips. But I think there's not enough motivation and money to do that.

Re: If Claude Fable stops helping you, you'll never know

#353

Post by llelouch »

nsingh2 wrote: post #217
jsw97 wrote: post #184

Given the high rate of false positives people are reporting for the non-silent cybersecurity, biological, etc., safeguards, there is a strong likelihood that you will encounter silently nerfed behavior even if you are _not_ violating their TOS. Ultimately this will be evident in the way customers / external benchmarkers experience Fable. Hopefully competition will drive future models toward a lower false positive rat…

It's such an obviously bad policy, it's mind-boggling that they thought this was a good idea. It just breeds paranoia and mistrust, especially when people are already a bit paranoid about silent model quantification for cost cutting reasons.

What's the alternative? Not release the model at all?

"Make the guardrails better" isn't very hard and probably not worth the effort.

Re: If Claude Fable stops helping you, you'll never know

#356

Post by novaomnidev »

gck1 wrote: post #241

Wait, so to get this straight, Anthropic knows: 1) LLMs are non-deterministic 2) This class of models has a particular tendency to "misbehave" 3) Their classifiers have a high rate of false positives 4) Millions of people give these models access to their machines And they still decided to specifically train this model to sabotage work if it thinks the work may be in competition with Anthropic? I think this has a nam…

That is the perfect description. malware! What is sad is that there is no going back from this. Now that we know that they do this, I'll never believe they aren't doing it in other domains, or won't extend it to other domains in the future. This is probably the worst thing they could have possibly ever done for trust.

Re: If Claude Fable stops helping you, you'll never know

#357

Post by theLiminator »

altcognito wrote: post #228
theLiminator wrote: post #151

Earlier quoted context omitted.

> But, history says the supercomputer of today will fit in your pocket in a few years. I don't think this will be true in the same time span anymore. Each miniaturization is costing more and more money. Perhaps they'll come up with exotic fundamental improvements, but I don't think the rate of improvement of compute/watt will match the previous decades.

Single clock speed hasn't had much of an upgrade, but the architecture for doing exactly what they are doing? That will improve for at least 5-10 years. There are both huge power gains from Processing in Memory (PIM) chips (70-80% discount in energy), and improvements to engineering to make memory cheaper and cheaper.

Yes, I'm talking about a supercomputer from today in your pocket. That probably requires at least 5000x perf/watt if not even more.

Re: If Claude Fable stops helping you, you'll never know

#358

Post by llelouch »

colechristensen wrote: post #330
afavour wrote: post #28

Earlier quoted context omitted.

The charitable read is that their restrictions for "safety" (i.e. what's separating Fable from Mythos) makes this inevitable. If you could just make your own Mythos it would circumvent the protection. Which kinda just highlights how weird this situation is.

"Safety" is just marketing spin for their coming attempt at monetization based on exclusivity.

I don't think you know the people working at anthropic. They truly believe this. I think someone people are so used to people like Altman they think everyone else is like him.

Re: If Claude Fable stops helping you, you'll never know

#359

Post by hypfer »

baq wrote: post #343
reissbaker wrote: post #248

Earlier quoted context omitted.

Many, many, many public policy positions; for a clear-cut example, they eventually supported SB 1047 [1] which would have banned open-sourcing any model trained with over 10^26 FLOPS (i.e. what Anthropic reportedly used to train Mythos). Their "Responsible Scaling Policy" [2] — a set of policy proposals that includes recommendations for government regulation — specifically calls out requiring "third-party controls" o…

The nuance is not what they propose, but why, even according to them, they propose this. Honestly the proposals are appalling, but biosafety arguments are not immediately dismissible. Ultimate cyber threats we can handle by rewinding society 50 years back. We can’t undo a novel genetically engineered virus.

I mean arguably, we _could_ create conditions in which it is much less likely for people to start developing such a novel genetically engineered virus.

If you think about the factors that lead to people wanting to do such a thing, they're almost always tied to (perceived) inequality, (perceived) injustice or similar in some way.

I do believe that we could greatly reduce a whole bunch of such risks by just stopping to squeeze people as hard as we do right now.

But that would require a major refactoring I guess.

Re: If Claude Fable stops helping you, you'll never know

#360

Post by diimdeep »

1990s: "What a computer is to me is it's the most remarkable tool that we have ever come up with. It's the equivalent of a bicycle for our minds."

2026: /s "What a LLM is to me is it's the most remarkable tool that we have ever come up with. It's the equivalent of a bicycle for our minds, but for your mind it's a rental unicycle that will break apart under you if you pedal towards your own bicycle factory"

This wanna be cloud feudal lord likes to imagine that AI access is not yet freely tradable good, and his virtual digital peasants must think that his prerogatives should be taken as given, while preventing his future vassals from building their own castles.

Post reply on HN