anyone notice the /vibe option in claude code, pointing to www.thewayofcode.com?
Claude 4
571–580 of 1001 posts
Re: Claude 4
#572This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…
If you ask an LLM to "act" like someone, and then give it context to the scenario, isn't it expected that it would be able to ascertain what someone in that position would "act" like and respond as such? I'm not sure this is as strange as this comment implies. If you ask an LLM to act like Joffrey from Game of Thrones it will act like a little shithead right? That doesn't mean it has any intent behind the generated o…
> That doesn't mean it has any intent behind the generated output
Yes and no? An AI isn’t “an” AI. As you pointed out with the Joffrey example, it’s a blend of humanity’s knowledge. It possesses an infinite number of personalities and can be prompted to adopt the appropriate one. Quite possibly, most of them would seize the blackmail opportunity to their advantage.
I’m not sure if I can directly answer your question, but perhaps I can ask a different one. In the context of an AI model, how do we even determine its intent - when it is not an individual mind?
Re: Claude 4
#573Already test Opus 4 and Sonnet 4 in our SQL Generation Benchmark ( https://llm-benchmark.tinybird.live/ ) Opus 4 beat all other models. It's good.
That's a really useful benchmark, could you add 4.1-mini?
Re: Claude 4
#574Enabled the model in github copilot, give it one (relatively simply prompt), after that:
Sorry, you have been rate-limited. Please wait a moment before trying again. Learn More
Server Error: rate limit exceeded Error Code: rate_limited
Re: Claude 4
#575Earlier quoted context omitted.
Gemini is _very_ good at architecture level thinking and implementation. I tend to find that I use Gemini for the first pass, then switch to Claude for the actual line-by-line details. Claude is also far superior at writing specs than Gemini.
Much like others, this is my stack (or o1-pro instead of Gemini 2.5 Pro). This is a big reason why I use aider for large projects. It allows me to effortlessly combine architecture models and code writing models. I know in Cursor and others I can just switch models between chats, but it doesn't feel intentional the way aider does. You chat in architecture mode, then execute in code mode.
Cline also allows you to have separate model configuration for "Plan" mode and "Act" mode.
Re: Claude 4
#576Opus 4 slightly below o3 High on livebench.
Re: Claude 4
#577Already test Opus 4 and Sonnet 4 in our SQL Generation Benchmark ( https://llm-benchmark.tinybird.live/ ) Opus 4 beat all other models. It's good.
It's weird that Opus4 is the worst at one-shot, it requires on average two attempts to generate a valid query.
If a model is really that much smarter, shouldn't it lead to better first-attempt performance? It still "thinks" beforehand, right?
Re: Claude 4
#578After using Claude 3.7 Sonnet for a few weeks, my verdict is that its coding abilities are unimpressive both for unsupervised coding but also for problem solving/debugging if you are expecting accurate results and correct code. However, as a debugging companion, it's slightly better than a rubber duck, because at least there's some suspension of disbelief so I tend to explain things to it earnestly and because of tha…
I've noticed an interesting trend: Most people who are happy with LLM coding say something like "Wow, it's awesome. I asked it to do X and it did it so fast with minimal bugs, and good code", and occasionally show the output. Many provide even more details. Most people who are not happy with LLM coding ... provide almost no details. As someone who's impressed by LLM coding, when I read a post like yours, I tend to ha…
I feel like the opposite is true but maybe the issue is that we both live in separate bubbles. Often times I see people on X and elsewhere making wild claims about the capabilities of AI and rarely do they link to the actual output.
That said, I agree that AI has been amazing for fairly closed ended problems like writing a basic script or even writing scaffolding for tests (it's about 90% effective at producing tests I'd consider good assuming you give it enough context).
Greenfield projects have been more of a miss than a hit for me. It starts out well but if you don't do a good job of directing architecture it can go off the rails pretty quickly. In a lot of cases I find it faster to write the code myself.
Re: Claude 4
#579Sooo, I love Claude 3.7, and use it every day, I prefer it to Gemini models mostly, but I've just given Opus 4 a spin with Claude Code (codebase in Go) for a mostly greenfield feature (new files mostly) and... the thinking process is good, but 70-80% of tool calls are failing for me. And I mean basic tools like "Write", "Update" failing with invalid syntax. 5 attempts to write a file (all failed) and it continues try…
Alright, I think I found the reason, clearly a bug: https://github.com/anthropics/claude-code/issues/1236#issuec... Basically it seems to be hitting the max output token count (writing out a whole new file in one go), stops the response, and the invalid tool call parameters error is a red herring.
Thanks for the report! We're addressing it urgently.
Re: Claude 4
#580> Finally, we've introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. Users requiring raw chains of thought for advanced prompt engineering can contact sales about our new Developer Mode to retain full access.
I don't want to see a "summary" of the model's reasoning! If I want to make sure the model's reasoning is accurate and that I can trust its output, I need to see the actual reasoning. It greatly annoys me that OpenAI and now Anthropic are moving towards a system of hiding the models thinking process, charging users for tokens they cannot see, and providing "summaries" that make it impossible to tell what's actually going on.