I'm probably not going to hit the weekly limit, but it makes me nervous that the limit is weekly as opposed to every 36 hours or something. If I do hit the limit, that's it for the entire week —a long time to be without a tool I've grown accustomed to! I feel like someone is going to reply that I'm too reliant on Claude or something. Maybe that's true, but I'd feel the same about the prospect of loosing ripgrep for a…
This is how I feel about the 100 msg/wk limit on o3 for the ChatGPT plus plan. There’s no way to see how much I’ve used, and it’s an important enough resource that my lizard brain wants to hoard it. The result is that I way underutilize my plan and go for one of the o4-mini models instead. I would much prefer a lower daily limit, but maybe the underutilization is the point of the weekly limit. *edited to change “pro”…
Not sure. But o3 seems to be 200/10 days, not weekly anymore in my opinion.
Possibly dumb suggestion, but what about adaptive limits? Option 1: You start out bursting requests, and then slow them down gradually, and after a "cool-down period" they can burst again. This way users can still be productive for a short time without churning your servers, then take a break and come back. Option 2: "Data cap": like mobile providers, a certain number of high requests, and after that you're capped to…
Definitely something telecom is doing. People seems to be ok with their “unlimited” plans that throttle after a usage cap. Actually curious how the economics of those plans work out.
OpenAI's "PRO" subscription is really a waste of money IMHO for this and other reasons. Decided to give PRO a try when I kept getting terrible results from the $20 option. So far it's perhaps 20% improved in complex code generation. It still has the extremely annoying ~350 line limit in its output. It still IGNORES EXPLICIT CONTINUOUS INSTRUCTIONS eg: do not remove existing comments. The opaque overriding rules that…
There are things it’s great at and things it deceives you with. In many things I needed it to check something for me I knew was a problem, o3 kept insisting it were possible due to reasons a,b,c, and thankfully gave me links. I knew it used to be a problem so surprised I followed the links only to read black on white it still wasn’t. So I explained to o3 that it’s wrong. Two messages later we were back at square one.…
The problem is that all current mainstream LLMs are autoregressive decoder-only, mostly but not exclusively transformers.
Their math can't apply modifiers like "this example/attempt there is wrong due to X,Y,Z" to anything that came before the modifier clause in the prompt.
Despite how enticing these models are to train, these limitations are inherent. (For this specific situation people recommend going back to just before the wrong output and editing the message to reflect this understanding, as the confidently wrong output with no advisory/correcting pre-clause will "pollute the context": the model will look at the context for some aspects coded into high(-er)-layer token embeddings, inherently can't include the correct/wrong aspect because we couldn't apply the "wrong"/correction to the confidently-wrong tokens, thus retrieves the confidently-wrong tokens, and subsequently spews even more BS.
Similar to how telling a GPT2/GPT3 model it's an expert on $topic made it actually be better on said topic, this affirmation of that the model made an error will prime the model to behave in a way that it gets yelled at again... sadly.)
I'm not sure how this will play out long term, but I really am not a fan of having to feel like I'm using a limited resource whenever I use an LLM. People like unlimited plans, we are used to them for internet, text messaging, etc. The current pricing models just feel bad.
Resources that are "Unlimited" in marketing speak are rate-limited in practice. Your unlimited internet connection limits your daily transfer by bandwidth, both at your port and at the remote service ports. Your daily amount of SMS is limited by sending rate. Your all-you-can-eat restaurant order is limited by your belly.
No wonder that access to an expensive API which is an LLM is also rate-limited.
What does surprise me is that you can't buy an extra serving by paying more (twice the limit for 3x the cost, for instance). Either subscriptions don't make enough money, or their limits are at their datacenters and they have no spare capacity for premium plans.
I'm probably not going to hit the weekly limit, but it makes me nervous that the limit is weekly as opposed to every 36 hours or something. If I do hit the limit, that's it for the entire week —a long time to be without a tool I've grown accustomed to! I feel like someone is going to reply that I'm too reliant on Claude or something. Maybe that's true, but I'd feel the same about the prospect of loosing ripgrep for a…
> Loosing it for a couple of days is more palatable
Sorry, I'll just be "that guy" for a moment.
Assuming that access is cut at a random time during the week, the average number of days without Claude would be 3.5.
That's not reasonable as it's dependant on usage.
So assume that you've always been just shy of hitting the limit, and you increase usage by 50%, then you'd hit the limit 4.67 days in. Just 2-3 hours shy of the weekend - a sort of reward for the week's increased effort.
It makes me sad that devs start relying on proprietary online services to be able to work. We have been lucky enough to have FOSS tools to do everything and not to have to rely on any specific service or company to work and some of us are deciding to become like Monsanto-addicted farmers who forgot how to do their jobs without something they have to pay every month.
It makes me sad that couriers start relying on third party wheeled machines like cars and motorbikes.
You ignore the fact that there are many suppliers there with plenty of competition.
The 'take me from A to N' is a pretty broad problem that can have many different solutions. Is that comparable?
I'm not sure how this will play out long term, but I really am not a fan of having to feel like I'm using a limited resource whenever I use an LLM. People like unlimited plans, we are used to them for internet, text messaging, etc. The current pricing models just feel bad.
Resources that are "Unlimited" in marketing speak are rate-limited in practice. Your unlimited internet connection limits your daily transfer by bandwidth, both at your port and at the remote service ports. Your daily amount of SMS is limited by sending rate. Your all-you-can-eat restaurant order is limited by your belly. No wonder that access to an expensive API which is an LLM is also rate-limited. What does surpri…
You can pay more. It's unlimited (sorta) through API at API pricing.
We can and did work without it. It just makes us many times faster. Nothing about using an LLM removes skills and abilities you already had before it.
>Nothing about using an LLM removes skills and abilities you already had before it. Funny story: The widespread of Knorr soup stock already made people unable to cook their own stock soup, or even worse, the skill to season their soup from just basic, fresh ingredients. Source: my mom.
I'm always surprised when people buy vegetable stock. So many people I know cook "from scratch" with base ingredients like stock out of a box.
And just as with cooking: most people won't care - and the same goes with LLMs. It can be good enough... Less efficient? Meh - cloud. AI slop image? Meh - cheaper than paying an artist. LLMs to get kids through school? Meh - something something school-of-life.
I look around and see many poorly educated people leaning hard into LLMs. These people are confusing parroting their prompt output as knowledge, especially in the education realm. And while LLMs may not "remove skills and abilities you already had before it" - you damn sure will lose any edge you had over time. It's a slippery slope of trading a honed skill for convenience. And in some cases that may be a worthwhile trade. In others that is a disaster waiting to happen.
That's a big if. People might feel faster because there is more "movement", but it's not clear if overall they are actually signicantly faster (though everyone would like us to believe so). And note that I objected online services, local LLM don't have the same issues.
People who use it extensively may know whether it makes them significantly faster or not.
There's 2 problems there
1. 'faster' is subjective since you cannot do the same task twice without the second time being biased by the learnings from the first pass
2. While speed might be one measure, I've rarely found speed to be the end goal. Unless you're writing something that's throw away, you'll be reading what was written many times over. Long term maintainability is at odds with speed, in most cases.