Looks like visual understanding of diagrams is improved significantly! For example, it was on par with Chat GPT 4o and Gemini 1.5 in parsing an ERD for a conceptual model, but now far excels over the others.
Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
351–360 of 758 posts
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#352Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#353Tried my standard go-to for testing, asked it to generate a voronoi diagram using p5js. For the sake of job security I'm relieved to see it still can't do a relatively simple task with ample representation in the Google search results. Granted, p5js is kind of niche, but not terribly so. It's arguably the most popular library for creating coding. In case you're wondering, I tried o1-preview, and while it did work, I…
It's surprising how much knowledge is not easily googleable and can only unearched by deep diving into OSS or asking an expert. I recently was debugging a rather naive gstreamer issue where I was seeing a delay in the processing. ChatGPT, Claude and Google were all unhelpful. I spend the next couple days reading the source code, found my answer, and thought it was a bug.
Asked the mailing list, and my problem was solved in 10 seconds by someone who could identify the exact parameter that was missing (and IMO, required some architecture knowledge on how gstreamer worked - and why the unrelatedly named parameter would fix it). The most difficult problems fall into this camp - I don't usually find myself reaching for LLMs when the problem is trivial unless it involves a mountain of boilerplate.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#354One of the funnier things during training with the new API (which can control your computer) was this: "Even while recording these demos, we encountered some amusing moments. In one, Claude accidentally stopped a long-running screen recording, causing all footage to be lost. Later, Claude took a break from our coding demo and began to peruse photos of Yellowstone National Park." [0] https://x.com/AnthropicAI/status/1…
This is, craaaaaazzzzzy. I'm just a layman, but to me, this is the most compelling evidence that things are starting to tilt toward AGI that I've ever seen.
Why are you surprised by LLMs doing irrational or weird things?
All machine learning models start off in a random state. As they progress through their training, their input/output pairs tend to mimic what they've been trained to mimic.
LLMs have been doing a great job mimicking our human flaws from the beginning because we train them on a ton of human generated data. Other weird behavior can be easily attributed to simple fact that they're initialized at a random state.
Being able to work on and prove non-trivial theorems is a better indication of AGI, IMO.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#355Earlier quoted context omitted.
In 2015, when I was asked by friends if I'm worried about Self driving Cars and AI, I answered: "I'll start worrying about AI when my Tesla starts listening to the radio because it's bored." ... that didn't take too long
Maybe that's why my car keeps turning on the music when I didn't ask -- I had always thought Tesla devs were just absolute noobs when it came to state management.
With state management implemented as sophisticated enough ML model, it stops being clear whether the noob is on the outside or inside of the system.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#356It's quite sad that application interoperability requires parsing bitmaps instead of exchanging structured information. Feels like a devastating failure in how we do computing.
You can blame normies for this. They love their ridiculous point and click (and tap) interfaces.
Fortunately, with function-calling (and recently, with guaranteed data structure), we've had access to application interoperability with LLMs for a while now.
Don't get mad at a company for developing for the masses - that's what they are expected to do.
But they built for us, first.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#357This is actually a huge deal. As someone building AI SaaS products, I used to have the position that directly integrating with APIs is going to get us most of the way there in terms of complete AI automation. I wanted to take at stab at this problem and started researching some daily busineses and how they use software. My brother-in-law (who is a doctor) showed me the bespoke software they use in his practice. Runni…
Basically, if it means companies can introduce automation without changing anything about the tooling/workflow/programs they already use, it's going to be MASSIVE. Just and install and a prompt and you've already automated a lengthy manual process - awesome.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#358It improves to 25.9 over the previous version of Claude 3.5 Sonnet (24.4) on NYT Connections: https://github.com/lechmazur/nyt-connections/ .
What a neat bench mark! I'm blown away that o1 absolutely crushes everyone else in this. I guess the chain of thought really hashes out those associations.
Isn't it possible that o1 was also trained on this data (or something super similar) directly? The score seems disproportionately high.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#359One of the funnier things during training with the new API (which can control your computer) was this: "Even while recording these demos, we encountered some amusing moments. In one, Claude accidentally stopped a long-running screen recording, causing all footage to be lost. Later, Claude took a break from our coding demo and began to peruse photos of Yellowstone National Park." [0] https://x.com/AnthropicAI/status/1…
I think the best use case for AI `Computer Use` would be a simple positioning of the mouse and asking for conformation before a click. For most use cases this is all people will want/need. If you don't know how to do something, it is basically teaching you how, in this case, rather than taking full control and doing things so fast you don't have time to stop of going rogue.
I totally agree with you. At orango.ai, we have implemented the auto-click feature, but before it clicks, we position the cursor on the button and display a brief loading animation, allowing the user to interrupt the process.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#360This needs more discussion: Claude using Claude on a computer for coding https://youtu.be/vH2f7cjXjKI?si=Tw7rBPGsavzb-LNo (3 mins) True end-user programming and product manager programming are coming, probably pretty soon. Not the same thing, but Midjourney went from v.1 to v.6 in less than 2 years. If something similar happens, most jobs that could be done remotely will be automatable in a few years.
> If something similar is the case, most jobs that can be done remotely will be automatable in a couple of years.
I'm really curious on the cost of that sort of thing. Seems astronomical atm, but as much as i get shocked at the today-cost, staffing is also a pretty insane cost.