Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 254 posts

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#121

Post by stilesja »

cheema33 wrote: post #80

> Fable-level results at 1/3 the cost I am guessing this is not targeting those of us on the heavily subsidized $200/mo plans. Sure, these plans may be temporary, but none of us really know how temporary they are. Until then, 1/3rd of the published API pricing is not very appealing.

I burned though my weekly fable usage last night on the $200 plan. I had $200 in promotional usage credits and was in the middle of executing a moderate sized coding plan. Ran on usage credits for about 1h 15m and burned $120 in usage credits. I was astounded to see how fast the $ usage added up. One problem was that I was using sub-agent execution so multiple agents were running simultaneously and I realized at the end that claude had "Forgotten" my directive to use cheaper models as appropriate for sub-agent tasks so I was running multiple instances of Fable at once. Still hard to imagine paying per token. $200 a month is high, $200 per night is crazy.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#122

Post by adam_rida »

blobbers wrote: post #119

I'm curious if there is measurable value in diversity of thought, and if there's diminishing returns on a single models thought pattern. For example, compute X tokens with model A, then feed those into model B, etc. to get chain of thought through a diverse set of mdoels rather than chain of thought through a heterogeneous chain. Humans seem to strongly believe echo chambers are bad. Are LLMs the same?

there are and this what you optimize for. ensemble learning has a long literature on this. you want models that have the most diverse pool of capabilities so they complement each other. in verifiable tasks or classification this is straightfroward but a bit tricker in open ended text gen or CoT. this is one of our core research question

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#124

Post by springtimesun »

I have been trying something like this with local only models. I think I’ve tried everything that will fit in 96GB alone and in concert with each other using real project data (Rails). I haven’t found much benefit to anything other than Qwen 3.6 27b dense feeding a Claude final pass. I really thought layering was going to work like the law of large numbers, but for my workload it just hasn’t.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#125

Post by combyn8tor »

gruez wrote: post #43
combyn8tor wrote: post #38

Earlier quoted context omitted.

You may want to consider the incomes of developers outside the US, students, unemployed. $200/month is a lot to a lot of people.

The point still stands. The chinese labs don't have super discounted plans, so if the price per task benchmarks[1] are correct, and we apply the discount, you'll actually be paying more by using cheaper chinese models and this technique. https://artificialanalysis.ai/agents/coding-agents#artificia...

The point doesn't stand, and this comment appears to be spam.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#126

Post by dannyw »

adam_rida wrote: post #15
kamranjon wrote: post #2

No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else. Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."

The evaluator is public here: https://echo.tracerml.ai/eval/ It currently exposes 907 stored rows across seven benchmark families, with prompts, outputs, grades, and cost records. More benchmarks are coming soon. Echo does not disclose its per-request routing decision because that policy is the product. We can, however, publish some of the eligible open-weight model pool, version dates, aggregate allocation mix, and…

> Echo does not disclose its per-request routing decision because that policy is the product

My honest advice: that's going to pull away a decent amount of potential customers, even though I think your idea/concept is fantastic.

For example, if we were to consider it for Canva, observerability and full transparency is critical requirement; we can't accept not knowing which model serves a request. Both for legal/contract reasons, co-ordinated capacity planning with API providers, or even just evaluating our prompts and harnesses; and debugging/tracing results that went wrong. So that renders it out of consideration; and also suggests some kind of adversarial relationship where customers aren't trusted with critical information.

I definitely understand you need to keep business value, but I don't think hiding which model a request is routed to, is the right one, or at least if you want to expand to bigger potential customers / more advanced LLM deployments.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#127

Post by ai-x »

onlyrealcuzzo wrote: post #82
glasss wrote: post #56

Earlier quoted context omitted.

Good ideas are usually still good across time and tools

Dogpile was only a good idea while Search Engines were mostly trash. You needed to search all of them to find something decent. That's roughly analogous to today. Ignoring cost, you'd be way better off asking all the LLMs to solve a problem (like coding) where you can verify the answer. So the question is, for things like that -> can a group of models perform better than frontier models, especially at a reasonable co…

There is no reason a smarter model will not build an internal smarter/efficient router itself

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#128

Post by blobbers »

adam_rida wrote: post #122
blobbers wrote: post #119

I'm curious if there is measurable value in diversity of thought, and if there's diminishing returns on a single models thought pattern. For example, compute X tokens with model A, then feed those into model B, etc. to get chain of thought through a diverse set of mdoels rather than chain of thought through a heterogeneous chain. Humans seem to strongly believe echo chambers are bad. Are LLMs the same?

there are and this what you optimize for. ensemble learning has a long literature on this. you want models that have the most diverse pool of capabilities so they complement each other. in verifiable tasks or classification this is straightfroward but a bit tricker in open ended text gen or CoT. this is one of our core research question

My experience is mostly around either ensembles of weak learners that are bootstrapped to create a strong learner, or around boosted learners where you're training on residuals. With CoT you're sort of adding entropy to your answer, but you're also possibly getting a more thorough answer?

I'm curious though if these training methods are convergent or are models actually different; just like how in the stock market people think they're "diversified" but the truth is their exposure is likely much more risk correlated than one might think.

In certain situations, one right answer is better than a committee discussing the problem, but in others its sometimes nice to have some alternative methods of solving something. Fun project nonetheless.

My approach to using multiple models has been less about CoT but more about time to first token, and how you can use a small model to start interacting with the user while in parallel the more complex model is building a larger more complex thought. My work on this was primarily for voice backed interfaces before the voice models became quite a lot faster.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#130

Post by byzantinegene »

cheema33 wrote: post #80

> Fable-level results at 1/3 the cost I am guessing this is not targeting those of us on the heavily subsidized $200/mo plans. Sure, these plans may be temporary, but none of us really know how temporary they are. Until then, 1/3rd of the published API pricing is not very appealing.

I believe they will last until they IPO, and not long after that. $200/mo plans are not good for their P&L when their users using $10000 worth api credits. That's -98% margin loss per user.
Post reply on HN