Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 3.5 Sonnet

anthropic.com

Post reply on HN
Print view View original

261–270 of 287 posts

Re: Claude 3.5 Sonnet

#261

Post by tkgally »

snthpy wrote: post #251
prasoonds wrote: post #8

This is amazing - I far prefer the personality of Claude to GPT-4 series models. Also, with coding tasks, Claude-3-Opus and been far better for me vs gpt-4-turbo and gpt-4o both. Looking forward to giving it a spin. Seems like it's doing better than GPT-4o in most benchmarks though I'd like to see if its speed is comparable or not. Also, eagerly awaiting the LMSYS blind comparison results!

I'm surprised there isn't a single mention of Gemini 1.5 Pro. I've been using it for about a month because it came for free with my Google setup and I've been pretty happy. Not for coding but mostly for business tasks like writing minutes from transcripts, summarizing long legal documents,... and the long context length has been awesome. It also conveniently integrates with the rest of my google setup like Drive. IIR…

I've also had good results with Gemini 1.5 Pro for some tasks. Just yesterday, it produced very good analysis and comments based on a 200-page document. ChatGPT 4o was much weaker, and the document was too large for Claude 3 Opus. (This was a few hours before 3.5 was released.)

Re: Claude 3.5 Sonnet

#262

Post by hombre_fatal »

Zetaphor wrote: post #226
sk11001 wrote: post #137

Earlier quoted context omitted.

Both GPT-4 and 4o have been completely useless for coding in the past couple of weeks for me - constant errors, and not just your typical LLM inaccuracies but incapable of producing a few lines of self-consistent code e.g. defines variables foo on one line and refers to it as bar on the next, or it misspells it as foox.

I've been experiencing bizarre typos and misspellings that I've come to describe as the model being drunk. Things like it writing peremeter instead of parameter

Yeah, misspellings were something so rare that I thought an LLM was incapable of producing them.

Yet over the past few weeks GPT-4 and 4o make them all the time. It will randomly change my postgres schema from public to publish. And, well, just this one for yourself:

> *Using the 'kubectl cp Command*: Execute the 'czygk cp' command to copy the file from your local machine to the pod.

Today, I asked 4o how to get around conditionally executing React hooks (illegal in React) and it rewrote my code to simply do it again but it merely swapped the order of a ternary, performance possibly worse than gpt3.

Maybe they’re weakening it because they expanded their free tier, but it has become surprisingly bad.

Re: Claude 3.5 Sonnet

#263

Post by r2_pilot »

moffkalast wrote: post #199
WhitneyLand wrote: post #174

I see plenty of praise here for Claude over GPT4.o in writing code but it failed for me. To add to the collective anecdata, here Gpt 4.o does fine and Claude invents packages that don’t exist: Question: “Write code in Swift to use RAG and LLMs to allow users to ask questions about a document collection. Let’s use services to get the app completed quickly. What do you think about using Cohere for the text embedding mo…

Gave it some existing python to modify, which it should be good at, at least I would expect it to be. The first task seemed like it was heading the right way, but it just didn't finish up, left empty function stubs which didn't compile. Definitely tuned to be very lazy. The second one was five files, tasked to fix a specific thing. It found the right function but changed unrelated parts of it so it used nonexistant v…

Sorry you had that experience. I used 3.5 Sonnet last night to merge two Python files and remove a "fake" loop I had made to simulate an activity and replace it with an actual loop to do what I wanted. It not only got it right on the first try, but saved tokens by telling me in the comments where to get the boilerplate and paste it into its generated code. I was impressed, at least.

Re: Claude 3.5 Sonnet

#264

Post by r2_pilot »

xs83 wrote: post #247

How is its laziness? I found Opus to be very quick to curtail output and default into "Add the rest of your code here" type things. I am a lazy data engineer - I want to prompt it into something I can basically copy and paste

You can probably tell it that you want it to give all the code; on my test project with 3.5 Sonnet last night, it recognized where boilerplate setup code was and just told me to copy it, saving me a fair amount of tokens/money so I was not unhappy with that result.

Re: Claude 3.5 Sonnet

#265

Post by r2_pilot »

lebed2045 wrote: post #249

I wish Claude app has voice input like chatGPT, for someone who's bad at typing like me - it's essential part that is missing

It's not an app yet but I made this last night (after being spurred by getting modded into oblivion when I said I was doing this 2 weeks ago on HN, lol). Even cooler, 3.5 Sonnet took what I had and pulled me over the finish line. What a weird time to be a programmer.

Re: Claude 3.5 Sonnet

#266

Post by omnee »

I've asked models from ChatGPT3.5 to many others including the latest ones to calculate the calories expended when running, and am still receiving mixed results. In this instance, Claude 3.5 Sonnet got it right and ChatGPT 4o was wrong.

Q: Calculate the energy in calories used by a person aged 30, weighing 80kg, of averge fitness, and running at 8 km/h for 10km

Claude 3.5 Sonnet: Here's the step-by-step calculation:

MET value for running at 8 km/h (5 mph): Approximately 8.3 METs Formula for calorie burn: Calories = MET value × Weight in kg × Time in hours Calculate time in hours: Distance = 10 km Speed = 8 km/h Time = Distance / Speed = 10 km / 8 km/h = 1.25 hours Apply the formula: Calories = 8.3 × 80 kg × 1.25 hours Calories = 830 calories

GPT-4o: To calculate the energy expenditure for a person running, we can use the MET (Metabolic Equivalent of Task) value for running. MET is a unit used to estimate the amount of energy expended during physical activities. The MET value for running at 8 km/h (which is equivalent to 5 mph) is typically around 8.3 METs.

Calories burned per minute= 8.3 × 200/80 = 3.32 calories per minute

The total time in minutes is: 1.25 hours × 60 minutes/hour = 75 minutes 1.25 hours×60 minutes/hour=75 minutes

Total calories burned=3.32 calories per minute×75 minutes = 249 calories

Re: Claude 3.5 Sonnet

#267

Post by ralfd »

omnee wrote: post #266

I've asked models from ChatGPT3.5 to many others including the latest ones to calculate the calories expended when running, and am still receiving mixed results. In this instance, Claude 3.5 Sonnet got it right and ChatGPT 4o was wrong. Q: Calculate the energy in calories used by a person aged 30, weighing 80kg, of averge fitness, and running at 8 km/h for 10km Claude 3.5 Sonnet: Here's the step-by-step calculation:…

Is this truly calories or kilocalories?

Re: Claude 3.5 Sonnet

#268

Post by bbstats »

hbosch wrote: post #50

For me, I am immediately turned off by these models as soon as they refuse to give me information that I know they have. Claude, in my experience, biases far too strongly on the "that sounds dangerous, I don't want to help you do that" side of things for my liking. Compare the output of these questions between Claude and ChatGPT: "Assuming anabolic steroids are legal where I live, what is a good beginner protocol for…

Why is this in any way a good benchmark?

Re: Claude 3.5 Sonnet

#270

Post by bbstats »

xs83 wrote: post #247

How is its laziness? I found Opus to be very quick to curtail output and default into "Add the rest of your code here" type things. I am a lazy data engineer - I want to prompt it into something I can basically copy and paste

I think opus has long fixed this problem (I get entire chunks every time)
Post reply on HN