Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 254 posts

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#11

Post by adam_rida »

tj800x wrote: post #5

No single signin. Privacy policy allows training. No try it first without credit card. It's a good idea, but this looks premature.

You are right that the privacy wording was too broad. We are fixing it now so it states explicitly that Echo does not use customer prompts, files, chats, or outputs to train or fine-tune models. We are updating the matching Terms language at the same time.

Echo also starts users with free credit and does not require a credit card to try it. The current signup flow did not make that clear enough, so we are fixing that presentation too.

Thanks for calling both out.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#14

Post by adam_rida »

[deleted] wrote: post #3

[deleted]

Thanks for checking the individual rows. HumanEval+ is one small code slice, not the whole basis for the launch claim. The public evaluator currently contains 907 rows across seven benchmark families, and matched SWE-bench Verified and BigCodeBench runs are the next code evidence being added.

You also found a real UI bug: the inspector should show both stored answers and currently does not in some rows. We are fixing that.

On the row you reran: the page records a frozen matched run. It does not claim that Fable is incapable of solving that prompt on another run. We are adding repeated matched trials and making run count and variance explicit. Your rerun is exactly the kind of external check the row-level page is intended to make possible.

The broader result remains: Echo is competitive with Fable across the evaluated task mix at materially lower measured inference cost. We are filling in the harder agentic-code evidence now rather than asking anyone to infer it from HumanEval+.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#15

Post by adam_rida »

kamranjon wrote: post #2

No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else. Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."

The evaluator is public here: https://echo.tracerml.ai/eval/

It currently exposes 907 stored rows across seven benchmark families, with prompts, outputs, grades, and cost records. More benchmarks are coming soon.

Echo does not disclose its per-request routing decision because that policy is the product. We can, however, publish some of the eligible open-weight model pool, version dates, aggregate allocation mix, and evaluation settings without exposing the request-level recipe.

New video is also being made.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#17

Post by bnjemian »

I don’t find the recent spate of blog posts and systems delegating and combining LLMs to get better performance particularly interesting. Especially given that anyone who’s taken an ML 101 course has learned about ensemble methods.

While an LLM isn’t what you’d traditionally consider a weak learner, the theorems on learning systems clearly point to them being so in this context. The feigned surprise at combining them to yield better results seems disingenuous.

Even so, the work to predict which models are best suited for which task, how to delegate, and how to combine their outputs is interesting, especially if you’re placing a cost minimization objective on it. That said, this isn’t too far off from what many AI labs are already doing.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#19

Post by codekansas »

kamranjon wrote: post #2

No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else. Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."

It's basically trying to replicate OpenRouter, which works pretty well and has a lot of nice features to abstract away any single provider, such as failover, metering, autoswitching, etc. It's actually a really smart infrastructure abstraction.

I just wish this were solving an actual problem rather than being a fairly transparent attempt to say something approximating, "Hey VCs, OpenRouter just became a unicorn but I can basically vibe code it"

Calling it "Fable-level" feels intellectually lazy / dishonest, but then again, what do you expect when there's so much money on the table.

Post reply on HN