is it really unthinkable that another oss/local model will be released by deepseek, alibaba, or even meta that once again give these companies a run for their money
This regression put Anthropic behind Chinese models actually.
Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
291–300 of 620 posts
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#292We dropped Claude. It's pretty clear this is a race to the bottom, and we don't want a hard dependency on another multi-billion dollar company just to write software We'll be keeping an eye on open models (of which we already make good use of). I think that's the way forward. Actually it would be great if everybody would put more focus on open models, perhaps we can come up with something like the "linux/postgres/git…
Another aspect I haven’t seen discussed too much is that if your competitor is 10x more productive with AI, and to stay relevant you also use AI and become 10x more productive. Does the business actually grow enough to justify the extra expense? Or are you pretty much in the same state as you were without AI, but you are both paying an AI tax to stay relevant?
Where's the evidence of competitors being 10x more productive? So far, everyone is simply bragging about how much code they have shipped last week, but that has zero relevance when it comes to productivity
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#293Earlier quoted context omitted.
You can argu that you will have skill atrophy by not using LLMs. We have gone multi cloud disaster recovery on our infrastructure. Something I would not have done yet, had we not had LLMs. I am learning at an incredible rate with LLMs.
You're learning at your standard rate of learning, you're just feeding yourself over-confidence on how much you're absorbing vs what the LLM is facilitating you rolling out.
This is such a weird statement in so many levels.
The latent assumption here is that learning is zero sum.
That you can take a 30 year old from 1856 bring them into present day and they will learn whatever subject as fast as a present day 20 year old.
That teachers doesn't matter.
That engagement doesn't matter.
Learning is not zero sum. Some cultural background makes learning easier, some mentoring makes is easier, and some techniques increases engagement in ways that increase learning speed.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#294Earlier quoted context omitted.
Any recommendations on good open ones? What are you using primarily?
LMArena actually has a nice Pareto distribution of ELO vs price for this model elo $/M --------------------------------------- glm-5.1 1538 2.60 glm-4.7 1440 1.41 minimax-m2.7 1422 0.97 minimax-m2.1-preview 1392 0.78 minimax-m2.5 1386 0.77 deepseek-v3.2-thinking 1369 0.38 mimo-v2-flash (non-thinking) 1337 0.24 https://arena.ai/leaderboard/code?viewBy=plot&license=open-s...
LMArena isn't very useful as a benchmark, however I can vouch for the fact that GLM 5.1 is astonishingly good. Several people I know who have a $100/mo Claude Code subscription are considering cancelling it and going all in on GLM, because it's finally gotten (for them) comparable to Opus 4.5/6. I don't use Opus myself, but I can definitely say that the jump from the (imvho) previous best open weight model Kimi K2.5 to this is otherworldly — and K2.5 was already a huge jump itself!
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#295Earlier quoted context omitted.
That's a lame attitude. There are local models that are last year's SOTA, but that's not good enough because this year's SOTA is even better yet still... I've said it before and I'll say it again, local models are "there" in terms of true productive usage for complex coding tasks. Like, for real, there. The issue right now is that buying the compute to run the top end local models is absurdly unaffordable. Both in ge…
I completely see your point, but when my / developer time is worth what it is compared to the cost of a frontier model subscription, I'm wary of choosing anything but the best model I can. I would love to be able to say I have X technique for compensating for the model shortfall, but my experience so far has been that bigger, later models out perform older, smaller ones. I genuinely hope this changes through. I under…
> I completely see your point, but when my / developer time is worth what it is compared to the cost of a frontier model subscription, I'm wary of choosing anything but the best model I can.
Don't you understand that by choosing the best model we can, we are, collectively, step by step devaluating what our time is worth? Do you really think we all can keep our fancy paychecks while keep using AI?
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#296Earlier quoted context omitted.
It is easy to not believe if you only apply an incredibly narrow world view. Open your eyes, and you might become a believer.
What is this, some sort of cult?
You mean the cult of "I can't see the viruses therefore they dint exist"? As in "I can't imagine something so it means it's a lie"?
Indeed, quite weird and no imagination.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#297Earlier quoted context omitted.
I've been assuming this for a while. If I have a complex feature, I use Opus 4.6 in copilot to plan (3 units of my monthly limit). Then have Grok or Gemini (.25-.33) of my monthly units to implement and verify the work. 80% of the time it works every time. Leave me plenty of usage over the month.
Yeah I've been arriving at the same thing. The other models give me way more usage but they don't seem to have enough common sense to be worth using as the main driver. If I can have Claude write up the plan, and the other models actually execute it, I'd get the best of both worlds. (Amusingly, I think Codex tolerates being invoked by Claude (de facto tolerated ToS violation), but not the other way around.)
I don't think there's any ToS violation involved? AIUI you can use GPT models with any harness, at least at present.
You could nonetheless have Codex write up the plan to an .md file for Claude (perhaps Sonnet or even Haiku?) to execute.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#298Earlier quoted context omitted.
No, it is an as snarky response to a person being snarky about usefulness of AI agents. It does seem like there is a cult of people who categorically see LLMs as being poor at anything without it being founded in anything experience other than their 2023 afternoon to play around with it.
Who cares? Why are people so invested in trying to “convert” others to see the light? Can’t you be satisfied with outcompeting “non believers”? What motivates you to argue on the internet about it? Deep down are you insecure about your reliance on these tools or something, and want everyone else to be as well?
Why do people invest themselves so hard in interjecting themselves into conversations about Ai telling people it doesn't work?
It feels so off rebuilding serious SaaS apps in days for production, only to be told it is not possible?
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#299AFAICT this uses a token-counting API so that it counts how many tokens are in the prompt, in two ways, so it's measuring the tokenizer change in isolation. Smarter models also sometimes produce shorter outputs and therefore fewer output tokens. That doesn't mean Opus 4.7 necessarily nets out cheaper, it might still be more expensive, but this comparison isn't really very useful.
For some real data, Artificial Analysis reported that 4.6 (max) and 4.7 (max) used 160M tokens and 100M tokens to complete their benchmark suite, respectively: https://artificialanalysis.ai/?intelligence-efficiency=intel... Looking at their cost breakdown, while input cost rose by $800, output cost dropped by $1400. Granted whether output offsets input will be very use-case dependent, and I imagine the delta is a lot…
This is the right way of thinking end-to-end.
Tokenizer changes are one piece to understand for sure, but as you say, you need to evaluate $/task not $/token or #tokens/task alone.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#300Earlier quoted context omitted.
Not so much atrophy as apathy. I've worked with people who will look at code they don't understand, say "llm says this", and express zero intention of learning something. Might even push back. Be proud of their ignorance. It's like, why even review that PR in the first place if you don't even know what you're working with?
Is this the same subset of people who copy/paste code directly from stack overflow without understanding ? I’m not sure this is a new problem.
It's difficult to copy & paste an entire app from Stack Overflow