Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Outsourcing plus local AI will soon become more economical vs. frontier labs

signalbloom.ai

Post reply on HN
Print view View original

141–150 of 408 posts

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#141

Post by logicchains »

zuzululu wrote: post #133

I keep seeing this narrative involving Deepseek as an example of OSS LLMs but they are subsidizing a huge amount of tokens at cost and one can easily understand why they are doing it if one is not lazy and think critically. It's still far too costly and not effective to use Local AI that can match what the frontier models can offer, especially when the inference hardware is being heavily restricted due to geopolitica…

>they are subsidizing a huge amount of tokens at cost

This is absolutely false, because other providers serving the Deepseek models on OpenRouter are also able to offer very low prices, and they don't have the money to subsidize anything.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#142

Post by EGreg »

Groxx wrote: post #42
EGreg wrote: post #16

Earlier quoted context omitted.

Oh, and neural networks doing a huge number of floating point operations per word is not energy-heavy? Training these neural networks every few months isn’t energy-heavy? Both Bitcoin and these large models weren’t “designed to be energy-heavy”. It was a consequence of first-gen design decisions to solve a specific problem. Then as time went on, costs went down and they became a huge outlier in terms of energy. The q…

Bitcoin is pretty much explicitly designed to use as much electricity as the market will allow, without becoming any more useful . If you removed 99% of the miners from the current system, Bitcoin will still be exactly the same - it won't be any faster or slower, and the same number of transactions will flow through. The cost of electricity serves only as a lower bound on the expected value of a coin. Neural nets on…

It wouldn’t be exactly the same, because if you had all that mining capacity and 99% magically took a holiday, there is now enough mining power to take over the network anytime. It’s not secure.

Same with AI. Now that the Mythos and other models are finding exploits in every code base and anyone can run them, you can’t afford anymore not to keep burning credits securing your code base. It’s like proof of work red queen theory. You have to run faster and faster just to stay in place. Great business model.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#143

Post by anthonypasq »

Forgeties79 wrote: post #123
anthonypasq wrote: post #114

Earlier quoted context omitted.

https://www.wsj.com/tech/ai/mind-blowing-growth-is-about-to-...

Let’s see it first. And without omitting training/infrastructure costs at that. Until then my comment is still accurate.

its a private company, what exactly do you expect to 'see'?

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#144

Post by cortesoft »

stingraycharles wrote: post #77
Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

Also, your local hardware is in no way capable of running the types of models that the cloud providers do, it’s just not economically feasible, and it never will be.

NEVER will be is a pretty big leap. Never is a long time.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#145

Post by zackify »

simonw wrote: post #79
Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

I learned today that the Anthropic "Enterprise" plan - the one big companies use because they need governance features and audit logs and all of that jazz - is billed at API token rates (plus $20/seat/month). So large companies are getting billed a lot more than those discount subscription plans.

We are on it at my job. It saves money due to other parts of the org not using as many tokens.

The real cost effective way is giving a team $20 cursor $20-100 Claude $20-200 codex.

I'm spending 1k on Claude enterprise easily and that's with trying to spread it on codex and cursor using pi.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#146

Post by throwa356262 »

zuzululu wrote: post #133

I keep seeing this narrative involving Deepseek as an example of OSS LLMs but they are subsidizing a huge amount of tokens at cost and one can easily understand why they are doing it if one is not lazy and think critically. It's still far too costly and not effective to use Local AI that can match what the frontier models can offer, especially when the inference hardware is being heavily restricted due to geopolitica…

Do you have a source for your first claim?

My impression is that deepseek designed v4 specifically for cheap inference and they are not loosing money even at 75% lower price.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#147

Post by dd8601fn »

stymaar wrote: post #95
alexriddle wrote: post #91

Earlier quoted context omitted.

Anything over 150 seats means you need to pay at token rates plus the $20/user. My day job is operational (no coding at all) and I'm spending ~$300 a month on a few chats with Claude/Cowork a day over the course of a month.

I hope your company is keeping the input/response pair in case they need to break free at some point.

Wouldn’t people mostly just want any artifacts?

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#148

Post by throwa356262 »

ianhxu wrote: post #26

>frontier models are more capable than the latest from DeepSeek. But is the capability difference enough to justify a 30x price difference? The contradiction here is that without frontier models, there'd be no foundation for models like DeepSeek to reference and catch up to. Is there an economic model that captures this kind of dynamic?

Hold on mate, do you realize that a significant number of recent major advances in AI came from deepseek?

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#149

Post by swader999 »

I'm finding sound judgment, common sense, technical depth and breadth, a feel for the UX are skills that amplify Agentic coding. Deep knowledge of the problem domain and time with the customer (or SME's or end users) are what build these. Outsourcing this will never work, you can't put someone 12 hours ahead of the timezone your serving in front of the customer.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#150

Post by ZeroCool2u »

A crucial factor tech industry folks tend to ignore is how much executives value predictable costs. Cloud migrations got away with this, but still had to argue fiercely, because 'the cloud' and its serverless tech had the potential to significantly decrease overall spend for unpredictable, bursty workloads.

The usual counter-argument is the operational burden, but human capital is also a relatively fixed cost. A dedicated team of 3-5 FTEs could probably handle inference ops for a F500 company.

Meanwhile, the capability delta is shrinking fast. We have more evidence that local open-source is viable with the release of DeepSeek v4, and the industry is only trending further in this direction. Especially as we rely more on test-time compute and task-specific harnesses rather than model size.

So, if you're an executive looking at a marginal but fixed operations cost, added flexibility, and a rapidly closing gap in capability, why wouldn't you just run open-source models on your own infrastructure to get those highly predictable costs? Plus, you decrease the risk of one of the frontier

Post reply on HN