Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Outsourcing plus local AI will soon become more economical vs. frontier labs

signalbloom.ai

Post reply on HN
Print view View original

111–120 of 408 posts

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#111

Post by runtime_terror »

wongarsu wrote: post #92
lelanthran wrote: post #54

Earlier quoted context omitted.

> When discussing LLM pricing, people are missing the plot. [ ... snipped ...] Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. And you think it is unreasonable to consider this unsustainable?

Depends on what their actual costs are. Either they are losing lots of money on subscriptions, or they make absolute bank on API pricing. Looking at the pricing of 1-2T models like Kimi or DeepSeek on the open market, I'm tempted to assume that inference costs are closer to subscription pricing than to API pricing. Especially considering that subscriptions a) distribute load over time via rate limits, and b) will inc…

> I'm tempted to assume that inference costs are closer to subscription pricing than to API pricing

So just going on vibes?

While some people don't like his content, Ed Zitron shows a lot of evidence for your assumption being very wrong.

These companies are bleeding cash at ungodly rates. It's likely their API pricing is still subsidized if you look at their overall financial picture.

Related, there's a good reason those API prices keep going up a lot every new version and it's not just because the models are better.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#112

Post by anthonypasq »

runtime_terror wrote: post #108
Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

> The subscription token price is 10x-40x cheaper than API pricing This is a temporary phenomenon. Expect either drastic price increases or draconian throttling or both in the coming months. These companies are operating at huge loses and have hundreds of billions in liabilities and commitments. They need to turn on the money faucet sooner than later.

Theres recent reporting that Anthropic will be profitable this quarter...

edit: I see in other comments on this thread you think Ed Zitron is a reliable pundit so that explains everything.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#113

Post by stymaar »

gpugreg wrote: post #102
stymaar wrote: post #93

Earlier quoted context omitted.

> Lastly, there is a massive difference in capabilities, determinism, and error handling between 5T SOTA models like Opus What's your source for Opus being a 5T model? > and tiny distillations from DeepSeek that perform well only in benchmarks. I don't think you know what you're talking about. Local models aren't “distillations from Deepseek”. And they don't perform well “only in benchmarks”, Qwen 3.6 is a very decen…

> What's your source for Opus being a 5T model? Elon Musk tweeted that Grok is 0.5T or 1/10th the size of Opus. https://xcancel.com/elonmusk/status/2042123561666855235#m While this source's reliability is certainly debatable, the size matches the results of this paper, in which researchers estimated the parameter count from model knowledge. https://01.me/research/ikp/

> While this source's reliability is certainly debatable

Massive understatement. Nowadays it has become hard to find a single Musk statement that doesn't contain at least one lie.

> the size matches the results of this paper, in which researchers estimated the parameter count from model knowledge. https://01.me/research/ikp/

Thanks for the pointer. This estimation has Grok 6 times bigger than Musk claims it is, so maybe that's where the lie is.

(I'm quite skeptical about that number though, it would be quite disappointing for the US tech if their flagship models had to be that much larger than the Chinese ones for such a small edge in performance. Because I don't think US labs are incompetent, I'd bet that US flagships aren't more than 2/3 times bigger than Chinese flagship. Otherwise it really doesn't bode well.)

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#114

Post by anthonypasq »

Forgeties79 wrote: post #100
wongarsu wrote: post #92

Earlier quoted context omitted.

Depends on what their actual costs are. Either they are losing lots of money on subscriptions, or they make absolute bank on API pricing. Looking at the pricing of 1-2T models like Kimi or DeepSeek on the open market, I'm tempted to assume that inference costs are closer to subscription pricing than to API pricing. Especially considering that subscriptions a) distribute load over time via rate limits, and b) will inc…

Considering not one company is in the black yet I don’t really know how we can say anyone is making bank, unless we want to count absurd levels of VC funding (now slowing down) I guess.

https://www.wsj.com/tech/ai/mind-blowing-growth-is-about-to-...

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#115

Post by runtime_terror »

spprashant wrote: post #55
treis wrote: post #28

I think this misses the forest for the trees. Working with ChatGPT is eerily similar to working with offshore Indian devs back in my enterprise days. Productive if guided explicitly but if let run wild there's lots of WTF moments. LLMs are likely to replace outsourced devs because your employees that know the context can use LLMs to do what offshore devs did before.

"offshore Indian devs" are no slouches. They have access to the same GPT models and likely cost a tenth of the median US salary. Businesses are always looking to lower marginal cost. They will hire 1 software architect in US to write specs and 10 software developers in India to babysit 100 agents.

Obviously this is just anecdotal but over my 20+ year career I've worked with a lot of outsourced teams in India and my experience has nearly always been that they require a frustratingly specific degree of direction to product anything of quality.

Just recently I asked a dev there for a POC of a feature with decent specificity and ended up with about 8k LOC of spaghetti. I re-wrote it later in a few hundred. This is about in-line with my career experience.

I've had a few standout devs there but it does feel like a lot are putting in the bare minimum or are just working really far outside of their abilities.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#116

Post by bachmeier »

stingraycharles wrote: post #77
Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

Also, your local hardware is in no way capable of running the types of models that the cloud providers do, it’s just not economically feasible, and it never will be.

Very much dependent on the situation. For many business tasks, local hardware is good enough. But what a lot of folks overlook when saying these things is that (a) workers do more than run AI models on a piece of hardware, (b) significant computer hardware is already sitting idle outside normal work hours, when it can be running batch jobs, and (c) employees can share local hardware.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#117

Post by ghrl »

henry2023 wrote: post #85
ecshafer wrote: post #67

I have really been trying to get local models to work. I have tried different harnesses, tooling, skills, prompts, etc. But when I compare claude code with anthropic models or codex with gpt 5.5, vs qwen, glm or gemma and the same harnesses, the frontier models come out massively ahead. I am at the point where I just don't see the point of the non-frontier models, they waste more time than they save.

local models are 3 to 6 months behind SOTA models with the huge benefit of not needing to send all your IP to a shady third party. If inference cost comes down (as it has been for the last few years) you’ll be able to run today’s SOTA in your laptop by the end of the year.

I would say that is highly unlikely if by SOTA models you are not just referring to coding benchmarks but more general purpose ability and domain-specific knowledge. For example Kimi 2.6, which is comparable to Opus 4.6, is roughly 500+GB large, and I don't see how that would run on consumer hardware anytime soon. Besides, this is not just about the technical feasibility, but also economically not viable whatsoever. Why should consumer laptops be capable of running such models, when they would be massively underutilized most of the time, when inference providers can produce the same results faster, cheaper and a lot more viable economically?

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#118

Post by alfiedotwtf »

runtime_terror wrote: post #108
Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

> The subscription token price is 10x-40x cheaper than API pricing This is a temporary phenomenon. Expect either drastic price increases or draconian throttling or both in the coming months. These companies are operating at huge loses and have hundreds of billions in liabilities and commitments. They need to turn on the money faucet sooner than later.

Incentives matter…

If prices keep going up, watch for companies to exit frontier models and go to local llama.cpp instances for 6-month-ago SOTA, with the flex of being housed within the office - no more privacy leakage, no more price gouging.

To be honest, I’m not sure why a Y-Combinator backed company hasn’t come out yet flooding the market with highly capable OPAI (pronounced “Oh-pah” as in what Greeks shout as the drink shots), which stands for “On-Prem AI”

… yes, I just made up OPAI right now lol

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#119

Post by Npovview »

runtime_terror wrote: post #108
Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

> The subscription token price is 10x-40x cheaper than API pricing This is a temporary phenomenon. Expect either drastic price increases or draconian throttling or both in the coming months. These companies are operating at huge loses and have hundreds of billions in liabilities and commitments. They need to turn on the money faucet sooner than later.

Even with increased prices, AI enables velocity both in development and bugs fixing. Would companies want that? If prices are biting the company, I think companies will route all development and bugs fixing requests through few superperfomer developers with complete knowledge of the different components within the company (they will be the Queen Bees holding the company on their head). The rest of the company will be tasked with requirment gathering, specs cleaning, deambiguation and so on (worker bees).

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#120

Post by xbmcuser »

Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

Its not like the non frontier are not improving. If someone can use deepseek to get 90% of the work done for $100 then pay another $100 to anthropic or openai to complete it I think they will rather do that than pay anthropic or openai for $1000.
Post reply on HN