Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 4

anthropic.com

Post reply on HN
Print view View original

701–710 of 1001 posts

Re: Claude 4

#701

Post by BeefySwain »

jjmarr wrote: post #568
GolDDranks wrote: post #437

After using Claude 3.7 Sonnet for a few weeks, my verdict is that its coding abilities are unimpressive both for unsupervised coding but also for problem solving/debugging if you are expecting accurate results and correct code. However, as a debugging companion, it's slightly better than a rubber duck, because at least there's some suspension of disbelief so I tend to explain things to it earnestly and because of tha…

As a junior developer it's much easier for me to jump into a new codebase or language and make an impact. I just shipped a new error message in LLVM because Cline found the 5 spots in 10k+ files where I needed to make the code changes. When I started an internship last year, it took me weeks to learn my way around my team's relatively smaller codebase. I consider this a skill and cost issue. If you are rich and able…

I'm curious what tooling you are using to accomplish this?

Re: Claude 4

#702

Post by deadbabe »

gmueckl wrote: post #620
GuB-42 wrote: post #594

Earlier quoted context omitted.

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

Isn't the ultimate irony in this that all these stories and rants about out-of-control AIs are now training LLMs to exhibit these exact behaviors that were almost universally deemed bad?

It’s not just AI. Human software engineers will read some dystopian sci-fi novel or watch something on black mirror and think “Hey that’s a cool idea!” and then go implement it with no regard for real world consequences.

Re: Claude 4

#703

Post by deadbabe »

DubiousPusher wrote: post #654
gmueckl wrote: post #620

Earlier quoted context omitted.

Isn't the ultimate irony in this that all these stories and rants about out-of-control AIs are now training LLMs to exhibit these exact behaviors that were almost universally deemed bad?

This is a phenomenon I call cinetrope. Films influence the world which in turn influences film and so on creating a feedback effect. For example, we have certain films to thank for an escalation in the tactics used by bank robbers which influenced the creation of SWAT which in turn influenced films like Heat and so on.

Maybe this is why American society, with the rich amount of media it produces and has available for consumption compared to other countries, is slowly degrading.

Re: Claude 4

#704

Post by nickandbro »

I am making a vim golf app, https://vimgolf.ai , for my side project where users compete with an AI vim opponent to learn vim and so far claude 4 sonnet (haven't tried opus) is ranked the highest in my evals against o3, o4-mini-high, and gemini 2.5 pro. After 3 retries, it solves this challenge:

https://www.vimgolf.com/challenges/5957c3356f7e17045b00002c

here is a snippet of its thinking process pinpointing the exact problem with getting it to successfully transform the start file to the end file:

So I need to add a space between "string" and the backtick.

Let me think about what's happening. In my commands:

1. `5G$a `xml:"field1"``` 2. `6G$a `xml:"field2"``` 3. `7G$a `xml:"field3"```

The issue might be that I'm not actually including a space character in my command. Let me be very explicit about this. I need to use `` to represent the space character.

Re: Claude 4

#705

Post by 0x457 »

UncleEntity wrote: post #302
james_marks wrote: post #17

> we’ve significantly reduced behavior where the models use shortcuts or loopholes to complete tasks. Both models are 65% less likely to engage in this behavior than Sonnet 3.7 on agentic tasks Sounds like it’ll be better at writing meaningful tests

In my experience, when presented with a failing test it would simply try to make the test pass instead of determining why the test was failing. Usually by hard coding the test parameters (or whatever) in the failing function... which was super annoying.

I once saw probably 10 iterations to fix a broken test, then it decided that we don't need this test at all, and it tried to just remove it.

IMO, you either write tests and let it write implementation or write implementation and let it write tests. Maybe use something to write tests, then forbid "implementor" to modify them.

Re: Claude 4

#706

Post by sidibe »

lwansbrough wrote: post #325
saaaaaam wrote: post #320

I've been using Claude Opus 4 the past couple of hours. I absolutely HATE the new personality it's got. Like ChatGPT at its worst. Awful. Completely over the top "this is brilliant" or "this completely destroys the argument!" or "this is catastrophically bad for them". I hope they fix this very quickly.

What's with all the models exhibiting sycophancy at the same time? Recently ChatGPT, Gemini 2.5 Pro latest seems more sycophantic, now Claude. Is it deliberate, or a side effect?

Don't downplay yourself! As the models get more advanced they are getting better at recognizing how amazing you are with your insightful prompts

Re: Claude 4

#707

Post by Tostino »

Aeolun wrote: post #663
GardenLetter27 wrote: post #412

Earlier quoted context omitted.

I've had issues with Godot and Rustls - where it gives code for some ancient version of the API.

> some ancient version of the API One and a half years old shudders

When everything it is trying to use is deprecated, yeah it matters.

Re: Claude 4

#708

Post by a10c »

nickandbro wrote: post #704

I am making a vim golf app, https://vimgolf.ai , for my side project where users compete with an AI vim opponent to learn vim and so far claude 4 sonnet (haven't tried opus) is ranked the highest in my evals against o3, o4-mini-high, and gemini 2.5 pro. After 3 retries, it solves this challenge: https://www.vimgolf.com/challenges/5957c3356f7e17045b00002c here is a snippet of its thinking process pinpointing the exact…

some feedback for your app. I'm just dumped on a screen and left to my own devices, zero hints, prompting etc.

At no point am I "learning vim through challenges"

Re: Claude 4

#709

Post by johnb231 »

GuB-42 wrote: post #594
briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

They emulate a complex human reasoning process in order to generate that text.

Re: Claude 4

#710

Post by tails4e »

GuB-42 wrote: post #594
briandw wrote: post #379

This is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further i…

When I see stories like this, I think that people tend to forget what LLMs really are. LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text. So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship…

Well doesnt this go somewhat to the root of consciousness? Are we not the sum of our experiences and reflections on those experiences? To say an LLM will 'simply' respond as would a character in a sorry about that scenario, in a way shows the power, it responds similarly to how a person would protecting itself in that scenario.... So to bring this to a logical conclusion, while not alive in a traditional sense, if an LLM exhibits behaviours of deception for self preservation, is that not still concerning?
Post reply on HN