Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 254 posts

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#51

Post by recursivegirth »

sscaryterry wrote: post #37
teruakohatu wrote: post #34

Earlier quoted context omitted.

Because that is a short term solution, it won’t be offered forever. Large organisations have to purchase credits at $/tokens. Eventually everyone else will too.

This is what OpenAI and Anthropic are trying to make everyone believe. Most accountants will flinch at this (they already are). The $200 odd plans are already out of reach of many, many people. The attrition of customers if they were to get rid of these subscriptions plans would be untenable.

I think you are looking at it incorrectly. No business is buying individual accounts, because if they do, they open themselves up to considerable risk.

The $200 plans are priced so that the power-users use them and then advocate about how great the product is. If you're buying a $200 plan, you're not doing it because of the price point but rather because of the amount of work it is doing for you.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#52

Post by lukan »

janalsncm wrote: post #23
yonatan8070 wrote: post #22

I'm not an expert on this, but this sounds a lot like a larger-scale MoE (Mixture of Experts) type of architecture. As I understand it, in an MoE model, you essentially have hundreds of smaller sub-models ("experts") that are good at different tasks, and for every generated token, a single "master" model chooses which ones are most relevant to participate, and you only activate them.

In MoE systems the routing decision is made per-token, not per prompt or task. It’s one of ML’s many confusing naming conventions. Even more confusingly, there are older pre-LLM MoE systems which ensemble and pool the predictions from multiple sub-components. For example in a random forest you could take the majority vote of the decision trees or the average of their numerical predictions. After that, we developed ne…

"In MoE systems the routing decision is made per-token, not per prompt or task."

Have there been experiments with doing it per task? Like, "oh this is python project, use this model" "oh this is about writing fantasy, use this"?

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#53

Post by WD-42 »

dimitrios1 wrote: post #47
kamranjon wrote: post #2

No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else. Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."

Unrelated - but reminds me of my favorite quote by tenderlove: "microservices turn function calls into distributed computing problems"

grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too

seem very confusing to grug

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#54

Post by eikenberry »

jacobgold wrote: post #33

> Fable-level results at 1/3 the cost using open-weight models But we get ~$2500/mo worth of Fable credits for $200/mo on Anthropic pan? I'm still confused why people (who don't have to use API billing) are chasing open weight models based on cost.

I think most people assume the subsidized plans will go away or get more limited eventually. They are basically a loss leader and a marketing cost that is very flexible and easy to change w/o directly impacting their primary customers.

Re: Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

#60

Post by trollbridge »

switchbak wrote: post #49
drnick1 wrote: post #46

Earlier quoted context omitted.

> When they are successful at making those illegal/inaccessible This would be like trying to outlaw Linux or peer-to-peer file sharing. It's technically possible to write and pass a law, but it's basically impossible to enforce it.

Enough to make it a non-started at the organizations that pay their bills. Everyone else isn't big enough to matter.

Going to be an interesting world where big enterprises have to spend 100X the cost for the same value of AI as startups and small businesses.
Post reply on HN