A professional tool is something that provides reliable and replicable results, LLMs offer none of this, and A/B testing is just further proof.
By that definition humans are not professional since we hallucinate and make mistakes all the time.
Please do not A/B test my workflow
11–20 of 225 posts
Re: Please do not A/B test my workflow
#12Section 6.b of the Claude Code terms says they can and will change the product offering from time to time, and I imagine that means on a user segment basis rather than any implied guarantee that everyone gets the same thing. b. Subscription content, features, and services. The content, features, and other services provided as part of your Subscription, and the duration of your Subscription, will be described in the o…
> To decompile, reverse engineer, disassemble, or otherwise reduce our Services to human-readable form, except when these restrictions are prohibited by applicable law.
Luckily, it doesn't seem like any service was reverse-engineered or decompiled here, only a software that lived on the authors disk.
Re: Please do not A/B test my workflow
#13They lose money at $200/month in most cases. Again, the old rules still apply. You are the product.
Re: Please do not A/B test my workflow
#14A professional tool is something that provides reliable and replicable results, LLMs offer none of this, and A/B testing is just further proof.
Anthropic have done a lot of things that would give me pause about trusting them in a professional context. They are anything but transparent, for example about the quota limits. Their vibe coded Claude code cli releases are a buggy mess too. Also the model quality inconsistency: before a new model release, there’s a week or two where their previous model is garbage.
A/B testing is fine in itself, you need to learn about improvements somehow, but this seems to be A/B testing cost saving optimisations rather than to provide the user with a better experience. Less transparency is rarely good.
This isn’t what I want from a professional tool. For business, we need consistency and reliability.
Re: Please do not A/B test my workflow
#15Section 6.b of the Claude Code terms says they can and will change the product offering from time to time, and I imagine that means on a user segment basis rather than any implied guarantee that everyone gets the same thing. b. Subscription content, features, and services. The content, features, and other services provided as part of your Subscription, and the duration of your Subscription, will be described in the o…
^ this, I was about to double check on it when I saw you did. None of these practices sound abnormal, maybe a little sketchy but that comes with using llms.
Re: Please do not A/B test my workflow
#16A professional tool is something that provides reliable and replicable results, LLMs offer none of this, and A/B testing is just further proof.
[flagged]
Re: Please do not A/B test my workflow
#17The ToS you agreed to gives Anthropic the right to modify the product at any time to improve it. Did you have your agent explain that to you, or did you assume a $200 subscription meant a frozen product?
I understand. Just with AI, I don't think the behavior should change so drastically. Which I understand is paradoxical because we enjoy it when it can 10x or 1000x our workflow. I think responsible AI includes more transparency and capability control.
You rent ai, you don’t own it (unless you self host).
Re: Please do not A/B test my workflow
#18Moved from CC to opencode a couple months ago because the vibes were not for me. Not bad per se but a bit too locked in and when I was looking at the raw prompts it was sending down the wire it was also quite lets call it "opinionated".
Plus things like not being able to control where the websearches go.
That said I have the luxury of being a hobbyist so I can accept 95% of cutting edge results for something more open. If it was my job I can see that going differently.
Re: Please do not A/B test my workflow
#19Section 6.b of the Claude Code terms says they can and will change the product offering from time to time, and I imagine that means on a user segment basis rather than any implied guarantee that everyone gets the same thing. b. Subscription content, features, and services. The content, features, and other services provided as part of your Subscription, and the duration of your Subscription, will be described in the o…
Why should anyone care about their TOS while they are laundering people’s work at a massive scale?
Re: Please do not A/B test my workflow
#20They lose money at $200/month in most cases. Again, the old rules still apply. You are the product.
>They lose money at $200/month in most cases.
Source? Every time I see claims on profitability it's always hand wavy justifications.