91–100 of 287 posts
On a first glance, CS3.5 appears to be slightly faster than gpt-4o (62 vs 49 tok/sec) and slightlhy less capable (78% vs 89% accuracy on our internal reasoning benchmark). When initially launched, gpt-4o had speed of over 100 tok/sec, surprised that speed went down as fast.
Anthropic is the new king. This isn't even Claude 3.5 Opus and it's already super impressive. The speed is insane. I asked it "Write an in depth tutorial on async programming in Go" and it filled out 8 sections of a tutorial with multiple examples per section before GPT4o got to the second section and GPT4o couldn't even finish the tutorial before quitting. I been a fan of Anthropic models since Claude 3. Despite the…
OpenAI must be cooking something huge for them to not be releasing products far ahead of competitors
Wouldn't be surprised if the only thing cooking is OpenAI itself.
Doesn't look to be available on Bedrock yet. Maybe tomorrow, since the article says June 21st also? We truly live in the future...
My go to test is the boat and goat: "You are on the shore of a river with a boat and a stoat. How do you get to the other side?"
Previous LLMs have pattern matched this example to the logic puzzle, and come up with a complicated scenario about going back and forth with the goat.
Claude 3.5 says to just bring the goat across in the boat, which is wrong but better than previous versions. So that's an improvement.
But when you add more objects in, Claude just gets weird with it. When I told Claude it had a "boat, a stoat, and a rope, a shovel, and a handkerchief" it decided it had to use all the things in the strangest way and advised me to drown a rodent:
1. Use the shovel to dig a small trench near the shore.
2. Fill the boat with water and sink it in the trench.
3. Place the stoat in the boat and cover it with the handkerchief to keep it calm.
4. Use the rope to pull the sunken boat across the river.
And that really summarizes my whole problem with LLMs -- if I were using this in a scenario that wasn't so obvious, say programming, I would have no idea steps 1-4 were nonsense. If the LLM doesn't know what's nonsense, and I don't know, then it's just the blind leading the blind.
Earlier quoted context omitted.
"I wish they'd implement branching conversations like in ChatGPT" Can you say more about this? I Google'd and I'm not finding much. I asked ChatGPT and its response was not the assumption I held about what "branching" meant [0]. [0] https://chatgpt.com/c/6b2e0f7c-c4e6-44df-9116-ac7f618200f2
I just mean that when you click the button to generate a new version of the response (or edit your own message), ChatGPT shows you the arrow buttons enabling you to go to the previous version of it, and that works for all the messages, so you can go back up a few messages and try a different version of the conversation, without losing what you've had before.
I remember having moments looking at the plans Opus generated and being impressed with its capabilities.
The slow speed of requests I could deal with, but the costs could quickly add up in workflows and the autonomous agent control loop. When GPT4o came out at half the price it made Opus quite pricey in comparison. I'd often thought if I could just have Opus capabilities at a fraction of the price, so its a nice surprise to have it here sooner that I imagined!
The agent platform isn't officially launched yet, but its up at https://github.com/trafficguard/nous
I never liked the Langchain API when I looked at the examples so I built it from scratch. It has an autonomous agent with a custom XML-based function calling, memory and call history. The software engineer agentic workflow I initially dog-fooded with a prompt like "Complete Jira XYZ-123". So it gets the Jira description, finds the appropriate terraform project in GitLab, clones, edits (delegating to Aider), creates a MR and messages on Slack. It also has a UI for running agents, human-in-the-loop interactions etc.
Opus remained better than GPT for me, even after the release of GPT-4o. VERY happy to see an even further improvement beyond that, Claude is a terrific product and given the news that GPT-5 only began its training several weeks ago I don't see any situation where Anthropic is dethroned in the near term. There are only two parts of Anthropic's offering I'm not a fan of: - Lack of conversation sharing: I had a conversa…
What I understand is that it's GPT 6 that just went into training, and that GPT 5 is complete and being delayed until after the U.S. election.
Opus remained better than GPT for me, even after the release of GPT-4o. VERY happy to see an even further improvement beyond that, Claude is a terrific product and given the news that GPT-5 only began its training several weeks ago I don't see any situation where Anthropic is dethroned in the near term. There are only two parts of Anthropic's offering I'm not a fan of: - Lack of conversation sharing: I had a conversa…
What I understand is that it's GPT 6 that just went into training, and that GPT 5 is complete and being delayed until after the U.S. election.