Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Outsourcing plus local AI will soon become more economical vs. frontier labs

signalbloom.ai

Post reply on HN
Print view View original

41–50 of 408 posts

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#41

Post by Aurornis »

jqpabc123 wrote: post #2

The current closed source frontier models are more capable than the latest from DeepSeek. But is the capability difference enough to justify a 30x price difference? "Frontier models" are caught in a financial dilemma of their own making --- they have spent such huge sums on development and as a result, they may have inadvertently priced themselves out of the market. Energy costs are a huge factor for AI. He who has t…

> they may have inadvertently priced themselves out of the market.

Last week we were all talking about how Anthropic has too much demand, how they had to rent a data center from a competitor, and how the limits they’ve put on their service to deal with the demand are making users angry.

DeepSeek is cheap because they’re working hard to attract users.

The open weights models released for free weren’t free to train. It’s a loss leader to get attention to try to sell you something in the future.

The prices we pay for tokens right now are set by supply and demand, with some being sold at high premiums and others at a loss. Some models are given away for free after the companies spent money on researchers and compute.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#42

Post by Groxx »

EGreg wrote: post #16
Hamuko wrote: post #9

Earlier quoted context omitted.

Bitcoin is a poor analogue for much anything since it's very much designed to be energy-heavy.

Oh, and neural networks doing a huge number of floating point operations per word is not energy-heavy? Training these neural networks every few months isn’t energy-heavy? Both Bitcoin and these large models weren’t “designed to be energy-heavy”. It was a consequence of first-gen design decisions to solve a specific problem. Then as time went on, costs went down and they became a huge outlier in terms of energy. The q…

Bitcoin is pretty much explicitly designed to use as much electricity as the market will allow, without becoming any more useful. If you removed 99% of the miners from the current system, Bitcoin will still be exactly the same - it won't be any faster or slower, and the same number of transactions will flow through. The cost of electricity serves only as a lower bound on the expected value of a coin.

Neural nets on the other hand generally show more capability as you add more compute power. There's a point where it's less valuable than the cost increase, so people don't do more than that, but it isn't constant value like Bitcoin.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#43

Post by chrisweekly »

rgbrenner wrote: post #29
dyauspitr wrote: post #20

Only if you don’t allow construction of local data centers

US has over 10x the number of data centers as China; and produces 2x more energy per capita than China.

what about energy consumption per capita?

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#44

Post by xcskier56 »

lumost wrote: post #34
treis wrote: post #28

I think this misses the forest for the trees. Working with ChatGPT is eerily similar to working with offshore Indian devs back in my enterprise days. Productive if guided explicitly but if let run wild there's lots of WTF moments. LLMs are likely to replace outsourced devs because your employees that know the context can use LLMs to do what offshore devs did before.

How many of those wtf moments are simply from not “being in the room when it happened?” Most enterprise software is riddled with wtf moments demanded as one compromise or another.

There's always wtf, why did we add this feature, but at least in my experience, once a week or so I run into something in this category. Me: "AI, please cleanup/refactor/improve this thing" AI: "Roger that! I deleted the file so now it's perfectly clean" ... insert W.T.F.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#45

Post by aftbit »

ianhxu wrote: post #26

>frontier models are more capable than the latest from DeepSeek. But is the capability difference enough to justify a 30x price difference? The contradiction here is that without frontier models, there'd be no foundation for models like DeepSeek to reference and catch up to. Is there an economic model that captures this kind of dynamic?

Free market competition? This is a pretty classic pattern. Leaders capture market with quality but run into trouble scaling, followers compete on price and availability. Given time, leaders eventually run out of upgrade runway and find themselves swallowed up by followers. Or alternatively, leaders think their lead is inevitable and miss a sea change or iterative upgrade path. Think IBM PCs before Compaq and other cheap clones ate their lunch.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#46

Post by GodelNumbering »

crimsoneer wrote: post #24

I think this is a compelling argument, but I think 2 issues: 1. I remain unconvinced LocalAI can work well for majority of businesses. It looks vaguely comparable on benchmarks, but it tends to be fragile and a lot of management overhead in reality. 2. Similarly, while Deepseek is comparable to Opus/Codex on benchmarks, for agentic work at scale I definitely notice the difference. That's not to say it's not economica…

Fair points. I used to think that until some months ago but the latest generation of OSS models are surprisingly good. Plus maybe it is the way I work, but I find myself constantly overriding the decisions of frontier LLMs (because they start degenerating towards god objects and spaghettification) so most use I have gotten out of the AI agents is really their ability to code quickly and syntactically correctly.

Also worth noting that it doesn't have to be full either-or, there can be a two tier enterprise deployment that routes to locally hosted vs frontier model, over time more and more usecases could get routed to local LLM

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#47

Post by aftbit »

crimsoneer wrote: post #24

I think this is a compelling argument, but I think 2 issues: 1. I remain unconvinced LocalAI can work well for majority of businesses. It looks vaguely comparable on benchmarks, but it tends to be fragile and a lot of management overhead in reality. 2. Similarly, while Deepseek is comparable to Opus/Codex on benchmarks, for agentic work at scale I definitely notice the difference. That's not to say it's not economica…

I wish Deepseek could read images. I've been having good luck guiding it around on personal projects, but anything that needs to render to a screen really needs to be looked at to see bugs.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#48

Post by Chyzwar »

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing.

The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team people that lack motivation and foundational skills.

Lastly, there is a massive difference in capabilities, determinism, and error handling between 5T SOTA models like Opus and tiny distillations from DeepSeek that perform well only in benchmarks.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#49

Post by hobofan »

crimsoneer wrote: post #24

I think this is a compelling argument, but I think 2 issues: 1. I remain unconvinced LocalAI can work well for majority of businesses. It looks vaguely comparable on benchmarks, but it tends to be fragile and a lot of management overhead in reality. 2. Similarly, while Deepseek is comparable to Opus/Codex on benchmarks, for agentic work at scale I definitely notice the difference. That's not to say it's not economica…

> It looks vaguely comparable on benchmarks, but it tends to be fragile and a lot of management overhead in reality.

I'm working on an self-hostable LLM (web) UI[0] that aims to provide a comparable good UX to e.g. ChatGPT, and you are right that there is a decent amount of fragility involved, and more management overhead than most people would expect.

However, we usually find that those details happen a lot more in e.g. the harness (= out application), or some prompt tuning that's required for each of the models, rather than model quality itself. We have seen customers using self-hosted LLMs with similar user satisfaction across their organization to other customers that heavily lean on latest GPT-5 models on Azure. Especially given that you have to do some level of tuning and setup anyways, you might as well invest it in "local"/self-hosted AI (if you can make the financials of the inference cost work out for you).

I think it should also be noted that the inference providers on hyperscalers also tend to be quite fragile, each in their own way (e.g. Google with a horrible rate limit system or Azure with almost weekly intermittent 500-error incidents).

[0]: https://github.com/EratoLab/erato

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#50

Post by marcusholt »

treis wrote: post #28

I think this misses the forest for the trees. Working with ChatGPT is eerily similar to working with offshore Indian devs back in my enterprise days. Productive if guided explicitly but if let run wild there's lots of WTF moments. LLMs are likely to replace outsourced devs because your employees that know the context can use LLMs to do what offshore devs did before.

[dead]
Post reply on HN