Ask HN: What is your experience with Claude 3.5?

But when you have complex code or a very long module, yes, Claude 3.5 performs a lot better.

ChatGPT is much more likely to miss small, but important details that can break the code in unexpected ways. And even when you point out the error to ChatGPT, it will double down on being confident in its solution and output exactly the same code again.

I've got to think Claude 3.5 had much better training data and training methodologies for code because it's much "smarter" at being logical and fitting pieces together into one coherent whole. For any random task, I'd trust Claude at coding 20-30% more than ChatGPT.

I only checked HN as I started getting service unavailable error pages in the middle of it helping me rewrite some code.

Only really tried it with embedded C and Next.js stuff and it can make a few unsafe mistakes but when I need a helper function it is much quicker than me finding one myself.

The part of the Claude UI that writes code in a sidebar next to the chat is much better than writing code inline like ChatGPT.

As a simple example from yesterday, we were generating a GitHub Action using GPT-4o and running into an issue.

The GitHub action does a diff of your code against `main` and then passes that diff to an LLM for a simple vulnerability analysis, returning a severity of high|medium|low plus a description.

Both GPT-4o and Claude 3.5 Sonnet kept getting caught up in the same problem, which is that the workflow they generated would log the contents of PR, which itself contained said workflow as YAML. This then caused the YAML to parse incorrectly.

A small example, but the issue was obvious to a developer reading the output. Both LLMs struggled to “understand” the issue, even when they were directly told what to do (they kept logging the diff for debug, which would continue to break it).

We got to a solution with both LLMs, but it required us to be quite explicit in our prompting.

Caveat: both GPT-4o and Claude 3.5 Sonnet were accessed via a self-hosted interface that hits the API directly, so your experience through the UIs may be different.