Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You only need the frontier model for one single edit

stencil.so

Post reply on HN
Print view View original

101–107 of 107 posts

Re: You only need the frontier model for one single edit

#101

Post by carterschonwald »

trollbridge wrote: post #25
carterschonwald wrote: post #21

Earlier quoted context omitted.

this so true. its really hard to make sure a model isnt going off the rails if i dont see full cot. the fact that oai and anthropic models hide it now has made them less reliable. which is a shame

The great news is you can use DS-V-Pro, MiMo-V2.5-Pro, GLM-5.2, or K3 and see everything. I tend to use 5.6-Sol now more for one shot type of tasks where all I want is the answer and I’m not going to read the reasoning.

5.6 sol is pretty good, its definitely very very well tuned, i'm not quite sure which of the others i should use near term, but theyre doing great work

Re: You only need the frontier model for one single edit

#102

Post by ricardobeat »

rolymath wrote: post #85
ricardobeat wrote: post #28

I just did this five minutes ago. Started a task with Kimi K3, noticed cost going up, switched to Minimax M3 continuing from the same context. Really easy to do with Crush/OpenCode. It works quite well most of the time. For very deeply technical tasks, where exploration involves multiple agents (and blowing up context limits a few times), I'll have a plan written down to disk that will be much longer than 2k tokens.

Wow. Someone using Crush in the wild. What do you think of it? I'm having trouble switching to it because the rest work fine, but would love to actually adopt it.

It has the most user-friendly UI of them all, a really usable scroll view. Automatically picks up LSPs. Has a well maintained provider registry. Looks nice and runs fast.

The main downside is that subagents cannot have edit tools, which kills any agent orchestration possibilities. I use it for most one-shot / single task prompts using open models.

Re: You only need the frontier model for one single edit

#103

Post by markust »

bensyverson wrote: post #95
markust wrote: post #79

Earlier quoted context omitted.

Perhaps, there's a small change that might work better here - if we use a different agent (by creating custom agents) for plan and implementation, then the switch to the implementation agent wouldn't end up re-reading a lot as the new agent would have fresh empty context. I implemented this[1] from one of the posters on HN and I think it'd work well with this problem too. Would be great if you could point out if my u…

The issue with this is just how much context it consumes. Generating the plan could easily get me to 30% of a 1M context window—the point at which I normally clear. Most of that context fill is not necessary for execution, and the model degrades as context fills. So starting from 30% (or passing a fresh agent all of that context) is not ideal. But maybe I'm misunderstanding what you're proposing?

Yes, sorry let me explain a bit more. The idea is that we have custom agents each having a model version of their own. So, if we have a planning agent that uses Opus and a worker agent that uses Sonnet, then the planning agent writes the plan and hands it over to the worker agent. This way, the handover here is not going to retain the context.

When the worker agent picks it up, it'd have a new context and only the todo list from the plan, so it shouldn't be reading all the files again.

Re: You only need the frontier model for one single edit

#104

Post by bensyverson »

markust wrote: post #103
bensyverson wrote: post #95

Earlier quoted context omitted.

The issue with this is just how much context it consumes. Generating the plan could easily get me to 30% of a 1M context window—the point at which I normally clear. Most of that context fill is not necessary for execution, and the model degrades as context fills. So starting from 30% (or passing a fresh agent all of that context) is not ideal. But maybe I'm misunderstanding what you're proposing?

Yes, sorry let me explain a bit more. The idea is that we have custom agents each having a model version of their own. So, if we have a planning agent that uses Opus and a worker agent that uses Sonnet, then the planning agent writes the plan and hands it over to the worker agent. This way, the handover here is not going to retain the context. When the worker agent picks it up, it'd have a new context and only the to…

Ah, got it. Yeah, that's what I do using my tool, Jobs. When the plan is big enough, it's definitely cheaper & more efficient than having the "big" agent do everything.

Re: You only need the frontier model for one single edit

#105

Post by can1357 »

robbie-c wrote: post #77

I'm begging people to write articles themselves rather than letting Claude do it for them. I want an expert opinion, if I just wanted to ask an LLM I have my own. How can this article not mention the KV cache even once?

The layout, as well as all of the paragraphs are written by me manually, although I generally draft a completely broken English one very rapidly and ask it to paraphrase, I do apologize if the editorial pass made it harder to read, but it really isn't an LLM's opinion :)

I thought KV cache not being as important was obvious, as a big part of the benchmrks were focused on pricing, but in retrospect I can see how I could've made that more clear.

In the end cost is what matters most and as seen from the results the KV cache miss on the cheap model doesn't really matter as much.

Re: You only need the frontier model for one single edit

#106

Post by can1357 »

hankbond wrote: post #11

Does anyone have a name for that really neat "the bar chart is a bookshelf, you can hover a spine to see the cover" vis? I've never seen it before.

Think I unironically spent more time on that than writing the dang thing. Glad you liked it though! Not sure if there's a name, was the best thing I could come up for visualizing turns.

Re: You only need the frontier model for one single edit

#107

Post by can1357 »

andreyvit wrote: post #41

It's hard to argue with the numbers, but starting with a (mostly true!) “research is the most expensive part” premise, this strikes me as an odd direction to go to optimize costs: 1. As others pointed out, we feed all the same research turns to a smaller model, so we pay the uncached price for all of them. 2. During research, the model typically reads more code than is relevant, to figure out what is relevant and wha…

I'd love nothing more than measure 3), but quality is really hard to assess IMO as even with the frontier models I find myself disagreeing on what is good and maintainable code; so that disqualifies LLM-as-judge, which even at it's best is somewhat RNG and costly.

+ I don't see this replacing anyones workflow to make it clear. This is mostly for fully autonomous agents that are given a task & are asked to execute. For instance, subagents are a perfect example where you can get the aforementioned savings for free, even if you don't run a "software factory" per-se, as you generally do not interact with them.

Post reply on HN