A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
1–10 of 37 posts
Re: A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
#2[deleted]
Re: A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
#3[deleted]
Re: A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
#4Seems interesting, but buried under a mountain of written slop.
Re: A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
#5Seems interesting, but buried under a mountain of written slop.
Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.
Re: A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
#6Seems interesting, but buried under a mountain of written slop.
Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.
Well I don’t know what the method is, there’s a mention of ironbee, but I’m not willing to spend the time digging through this to figure out what’s needed and how to set it up.
Re: A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
#7[deleted]
Re: A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
#8From what I understood, the gist of it is that uses a bunch of skills in a loop to help LLMs solve the tasks better via browser-testing, debugging, visual-testing, etc..
The skills can be found here:
https://github.com/ironbee-ai/ironbee-devtools-skills/tree/m...
Question that was left unanswered:
How much longer does a 4x DeepSkeep loop took compared to Opus?
Re: A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
#9Earlier quoted context omitted.
Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.
Well I don’t know what the method is, there’s a mention of ironbee, but I’m not willing to spend the time digging through this to figure out what’s needed and how to set it up.
You could literally send it to an ai
Re: A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost
#10Seems interesting, but buried under a mountain of written slop.
It is marketing for some SaaS thing. However, I think something like this, "verification loops with some sort of tricks", is what Anthropic and "OpenAI" do behind the scene with their interfaces.