It's quite sad that application interoperability requires parsing bitmaps instead of exchanging structured information. Feels like a devastating failure in how we do computing.
It's very much in the "worse is better" camp.
Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
241–250 of 758 posts
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#242Earlier quoted context omitted.
Because its a finetune of 3.5 optimized for the use case of computer use. Its actually accurate and its not a 3.6.
I don't think that's correct. This looks like a new model. Significant jump in math and gpqa scores.
Could be just additional post-training (aka finetuning) for coding/etc.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#243It's quite sad that application interoperability requires parsing bitmaps instead of exchanging structured information. Feels like a devastating failure in how we do computing.
See https://github.com/OpenAdaptAI/OpenAdapt for an open source alternative that includes operating system accessibility API data and DOM information (along with bitmaps) where available.
We are also planning on extracting runtime information using COM/AppleScript: https://github.com/OpenAdaptAI/OpenAdapt/issues/873
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#244It's quite sad that application interoperability requires parsing bitmaps instead of exchanging structured information. Feels like a devastating failure in how we do computing.
Apps are built for people rather than computers.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#245Great progress from Anthropic! They really shouldn't change models from under the hood, however. A name should refer to a specific set of model weights, more or less. On the other hand, as long as its actually advancing the Pareto frontier of capability, re-using the same name means everyone gets an upgrade with no switching costs. Though, all said, Claude still seems to be somewhat of an insider secret. "ChatGPT" ha…
There was a recent article[0] trending on HN a about their revenue numbers, split by B2C vs B2B. Based on it, it seems like Anthropic is 60% of OpenAI API-revenue wise, but just 4% B2C-revenue wise. Though I expect this is partly because the Claude web UI makes 3.5 available for free, and there's not that much reason to upgrade if you're not using it frequently. [0]: https://www.tanayj.com/p/openai-and-anthropic-reve…
I suppose business customers are savvy and will do enough research to find the best cost-performance LLM. Whereas consumers are more brand and habit oriented.
I do find myself running into Claude limits with moderate use. It's been so helpful, saving me hours of debugging some errors w/ OSS products. Totally worth $20/mo.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#246my quick notes on Computer Use: - "computer use" is basically using Claude's vision + tool use capability in a loop. There's a reference impl but there's no "claude desktop" app that just comes with this OOTB - they're basically advertising that they bumped up Claude 3.5's screen vision capability. we discussed the importance of this general computer agent approach with David on our pod https://x.com/swyx/status/1771…
See https://github.com/OpenAdaptAI/OpenAdapt for an open source implementation that includes a desktop app OOTB.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#247Great progress from Anthropic! They really shouldn't change models from under the hood, however. A name should refer to a specific set of model weights, more or less. On the other hand, as long as its actually advancing the Pareto frontier of capability, re-using the same name means everyone gets an upgrade with no switching costs. Though, all said, Claude still seems to be somewhat of an insider secret. "ChatGPT" ha…
> Great progress from Anthropic! They really shouldn't change models from under the hood, however. A name should refer to a specific set of model weights, more or less. In the API ( https://docs.anthropic.com/en/docs/about-claude/models ) they have proper naming you can rely on. I think the shorthand of "Sonnet 3.5" is just the "consumer friendly" name user-facing things will use. The new model in API parlance would…
That's great to know - business customers require a lot more stability, I suppose!
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#248Claude's current ability to use computers is imperfect. Some actions that people perform effortlessly—scrolling, dragging, zooming—currently present challenges for Claude and we encourage developers to begin exploration with low-risk tasks. Nice, but I wonder why didn't they use UI automation/accessibility libraries, that have access to the semantic structure of apps/web pages, as well as accessing documents directly…
We use operating system accessibility APIs when available in https://github.com/OpenAdaptAI/OpenAdapt.
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#249Claude is absurdly better at coding tasks than OpenAI. Like it's not even close. Particularly when it comes to hallucinations. Prompt for prompt, I see Claude being rock solid and returning fully executable code, with all the correct imports, while OpenAI struggles to even complete the task and will make up nonexistent libraries/APIs out of whole cloth.
I've been using a lot of o1-mini and having a good experience with it. Yesterday I decided to try sonnet 3.5. I asked for a simple but efficient script to perform fuzzy match in strings with Python. Strangely, it didn't even mention existing fast libraries, like FuzzyWuzzy and Rapidfuzz. It went on to create everything from scratch using standard libraries. I don't know, I thought this was something basic for it to s…
just ask it to use libraries you want; you cant expect it to magically read your mind, you need to guide every LLM to what are your must/nice haves
Re: Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
#250I have been a paying ChatGPT customer for a long time (since the very beginning). Last week I've compared ChatGPT to Claude and the results (to my eye) were better, the output better structured and the canvas works better. I'm on the edge of jumping ship.
I jumped ship in April of this year and haven’t looked back.
Use the best tool available for your needs. Don’t get trapped by a feeling of sunk cost.