It improves to 25.9 over the previous version of Claude 3.5 Sonnet (24.4) on NYT Connections: https://github.com/lechmazur/nyt-connections/.
Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
261–270 of 758 posts
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#262Earlier quoted context omitted.
It can actually scroll.
While we expect this capability to improve rapidly in the coming months, Claude's current ability to use computers is imperfect. Some actions that people perform effortlessly—scrolling, dragging, zooming—currently present challenges for Claude and we encourage developers to begin exploration with low-risk tasks.
Can someone please try this on a MAC/OS and just 100% verify if this puppy can scroll or not? thnks
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#263Not that I'm scared of this update but I'd probably be alright with pausing llm development today, atleast in regard to producing code.
I don't want an llm to write all my code, regardless of if it works, I like to write code. What these models are capable of at the moment is perfect for my needs and I'd be 100% okay if they didn't improve at all going forward.
Edit: also I don't see how an llm controlled system can ever replace a deterministic system for critical applications.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#264Both new Sonnet and gpt-4o still fail at a simple: "How many w's are in strawberry?" gpt-4o: There are 2 "w's" in "strawberry." Claude 3.5 Sonnet (new): Let me count the w's in "strawberry": 0 w's. (same question with 'r' succeeds) What is artificial about current gen of "artificial intelligence" is the way training (predict next token) and benchmarking (overfitting) is done. Perhaps a fresh approach is needed to ach…
It's bad at directly working on classical computer problems like math and data processing. But you can do it indirectly by having it write a program that produces the correct result. Interestingly, I didn't even have to have it run the program, although usually you would write a tool which counts the number of w's in "strawberry" and return the result Which produced: Here's a simple Python function that counts the nu…
I always thought the halting problem was an academic exercise, but here we see a potential practical use case. Actually this seems pretty dangerous letting the LLM write and automatically execute code. How good is the sandbox? Can I trick the LLM into writing a reverse shell and opening it up for me?
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#265Earlier quoted context omitted.
I have to agree. I've been chatting with Claude for the first time in a couple days and while it's very on-par with ChatGPT 4o in terms of capability, it has this difficult-to-quantify feeling of being warmer and friendlier to interact with. I think the human name, serif font, system prompt, and tendency to create visuals contributes to this feeling.
Probably, people find Claude's color palette warmer and inviting as well. I believe I do. But Claude definitely has few authentication hoops than chatgpt.com. Gemini has by far the least frequent authentication interruptions than the 3 models.
Well, it is extremely similar to that of Hacker News'.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#266Earlier quoted context omitted.
"computer use" is also as bad a marketing choice as possible for something that actually seems pretty cool.
I'm not sure what a better term is. It's kind of understated to me. An AI that can "use a computer" is a simple straightforward sentence but with wild implications.
[deleted]
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#267Earlier quoted context omitted.
The real problem with Claude for me currently is that it doesn't have full LaTeX support. I use AI's pretty much exclusively to assist with my school work (there's only so many hours in a day and one professor doesn't do his own homeworks before he assigns them) so LaTeX is essential. With that known, my experience is that ChatGPT is much friendlier. The Claude interface is clunkier and generally less helpful to me.…
you can enable latex support in the settings of Claude
Where? I see barely any settings in settings. Maybe it is not available for everyone, or maybe it depends on your answer to "What best describes your work?" (I have not tested).
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#268aider benchmarks for claude 3.5 new are impressive. From 77.4% to 83.5% beating o1-preview.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#269It's quite sad that application interoperability requires parsing bitmaps instead of exchanging structured information. Feels like a devastating failure in how we do computing.
It's really more of a reflection on where we're at in the timeline of computing, with humans having been the major user of apps and webs site up until now. Obviously we've had screen scraping and terminal emulation access to legacy apps for a while, and this is a continuation of that.
There have been, and continue to be, computer-centric ways to communicate with applications though, such as Windows COM/OLE, WinRT and Linux D-Bus, etc. Still, emulating human interaction does provide a fairly universal capability.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#270Earlier quoted context omitted.
actually, that's what makes chat gpt powerful. I like an LLM willing to go along with what ever I am trying to do, because one day I might be coding, and another day I might be just trying to role play, write a book, what ever. I really cant understand what you were expecting, a tool works with how you use it, if you smack a hammer into your face, don't complain about a bloody nose. maybe dont do like that?
So if you're trying to write code and mistakenly ask it how to use a nonexistent API, you'd rather it give you garbage rather than explaining your mistake and helping you fix it? After all, you're clearly just roleplaying, right?
[deleted]