Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Outsourcing plus local AI will soon become more economical vs. frontier labs

signalbloom.ai

Post reply on HN
Print view View original

201–210 of 408 posts

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#201

Post by nicoburns »

alfiedotwtf wrote: post #118
runtime_terror wrote: post #108

Earlier quoted context omitted.

> The subscription token price is 10x-40x cheaper than API pricing This is a temporary phenomenon. Expect either drastic price increases or draconian throttling or both in the coming months. These companies are operating at huge loses and have hundreds of billions in liabilities and commitments. They need to turn on the money faucet sooner than later.

Incentives matter… If prices keep going up, watch for companies to exit frontier models and go to local llama.cpp instances for 6-month-ago SOTA, with the flex of being housed within the office - no more privacy leakage, no more price gouging. To be honest, I’m not sure why a Y-Combinator backed company hasn’t come out yet flooding the market with highly capable OPAI (pronounced “Oh-pah” as in what Greeks shout as th…

> If prices keep going up, watch for companies to exit frontier models and go to local llama.cpp instances for 6-month-ago SOTA, with the flex of being housed within the office - no more privacy leakage, no more price gouging.

That or just hiring people to do the work! I hear rumours that this is already starting to happen in some places (perhaps those that were a little overzealous with AI-hype driven layoffs).

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#203

Post by janpeuker »

jyounker wrote: post #182

The problem with outsourcing, as opposed to remote developers, is that it takes a really good manager and tech lead to make it work. My experience is that you have to write extremely detailed design documents and work specifications in order to get effective results. These generally have to be as detailed as most effective prompts. Once you've written specs that detailed, why do you need outsourced developers and fro…

Outsourcing usually gives you exactly what you pay for, arguably more transparently than other ways. It’s just that transparency (i.e. the price for quality) is sometimes not passed on from management / procurement taking that decision down to the team eventually having to work in a distributed fashion.

I think that’s also where the assumptions of the original post are off - the difference between DeepSeek and a frontier model is not usually what low quality outsourcing can cover. So you probably end up paying a highly qualified outsourced engineer who may not be significantly cheaper (most outsourcing is not just due to cost but capacity and capability).

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#205

Post by htrp »

simonw wrote: post #79
Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

I learned today that the Anthropic "Enterprise" plan - the one big companies use because they need governance features and audit logs and all of that jazz - is billed at API token rates (plus $20/seat/month). So large companies are getting billed a lot more than those discount subscription plans.

> I learned today that the Anthropic "Enterprise" plan - the one big companies use because they need governance features and audit logs and all of that jazz - is billed at API token rates (plus $20/seat/month).

Can large enterprises just not use the API ? I have audit logs and what seem to be enterprise features through my anthropic account (platform.claude.ai)

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#206

Post by Our_Benefactors »

sourcecodeplz wrote: post #139
ghrl wrote: post #117

Earlier quoted context omitted.

I would say that is highly unlikely if by SOTA models you are not just referring to coding benchmarks but more general purpose ability and domain-specific knowledge. For example Kimi 2.6, which is comparable to Opus 4.6, is roughly 500+GB large, and I don't see how that would run on consumer hardware anytime soon. Besides, this is not just about the technical feasibility, but also economically not viable whatsoever.…

It runs right now on 512gb RAM Macs and PCs.

It runs like shit though in terms of tokens/second and still has a reduced context window. Vs a single claude prompt can easily get into 300k tokens without breaking a sweat.

I want local AI to be a thing but the hardware isn’t here yet, because the only options are a Mac Studio or DGX machines strapped together. RAM prices needs to crash before local AI has a chance at actually competing.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#207

Post by illusive4080 »

CuriouslyC wrote: post #185

The future of American frontier AI isn't API calls, it's you taking your task to OAI/Anthropic like a consultant/external entity, then getting a product or whatever back, without ever seeing a large volume of intermediate work. This is inevitable because of the combination of distillation threat and proprietary harness development effort required to push performance at the bleeding edge. OAI/Anthropic are 100% going…

No, because handing a project over the wall almost always ends in disaster. The requirements are never clear enough.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#208

Post by SoftTalker »

jyounker wrote: post #182

The problem with outsourcing, as opposed to remote developers, is that it takes a really good manager and tech lead to make it work. My experience is that you have to write extremely detailed design documents and work specifications in order to get effective results. These generally have to be as detailed as most effective prompts. Once you've written specs that detailed, why do you need outsourced developers and fro…

> you have to write extremely detailed design documents

Luckily LLMs can do that too.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#209

Post by illusive4080 »

eikenberry wrote: post #183
ecshafer wrote: post #67

I have really been trying to get local models to work. I have tried different harnesses, tooling, skills, prompts, etc. But when I compare claude code with anthropic models or codex with gpt 5.5, vs qwen, glm or gemma and the same harnesses, the frontier models come out massively ahead. I am at the point where I just don't see the point of the non-frontier models, they waste more time than they save.

The point is to not indenture yourself to a corporation whose motives do not align with your own.

Their motives are to make the best product to compete in a very competitive market.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#210

Post by jqpabc123 »

tpolm wrote: post #78
dboreham wrote: post #75

Earlier quoted context omitted.

Cheap.

What is expensive in nuclear energy? Reason there is not more of nuclear reactors is not the cost, it is regulation. Regulation can be changed (it also seem to already have, recently, IIRC - starting 2024 NRC law changes by Biden admin and later by Trump admin)

Reason there is not more of nuclear reactors is not the cost, it is regulation.

The reason for regulation is that failure is not an option --- unless you're willing to accept the cost of making a big chunk of a state uninhabitable for a very long time.

How much would failure cost? The Chernobyl exclusion zone is over 1000 square miles --- about half the size of Delaware. And it is expected to remain uninhabitable for the next 20,000 years.

Also, the Russian Academy of Sciences estimates that up to 1 million people may suffer premature death as a result of radiation exposure and contamination from the event.

In the long run, renewable energy is a lot cheaper.

Post reply on HN