Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Outsourcing plus local AI will soon become more economical vs. frontier labs

signalbloom.ai

Post reply on HN
Print view View original

11–20 of 408 posts

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#11

Post by pjmlp »

jqpabc123 wrote: post #2

The current closed source frontier models are more capable than the latest from DeepSeek. But is the capability difference enough to justify a 30x price difference? "Frontier models" are caught in a financial dilemma of their own making --- they have spent such huge sums on development and as a result, they may have inadvertently priced themselves out of the market. Energy costs are a huge factor for AI. He who has t…

Energy costs and privacy.

Currently the projects I am involved require devs to use approaches like Ollama, Foundry Local and co if they happen to have good enough hardware, picking the best alternatives out of https://www.canirun.ai.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#12

Post by treis »

jqpabc123 wrote: post #2

The current closed source frontier models are more capable than the latest from DeepSeek. But is the capability difference enough to justify a 30x price difference? "Frontier models" are caught in a financial dilemma of their own making --- they have spent such huge sums on development and as a result, they may have inadvertently priced themselves out of the market. Energy costs are a huge factor for AI. He who has t…

Historically the winners in software have a flywheel that turns faster with more users. Facebook the more of your friends on it the better the product was. Google tracked how long users were on pages to improve search.

The frontier models are going to win that way. They won't feed your code back into the system but they will track which code you keep and what code gets a "try again claude".

They're not going to lose on price. No consumer software ever has because ultimately it's not that expensive relative to salary and the marginal cost is 0.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#14

Post by GodelNumbering »

andsoitis wrote: post #5
GodelNumbering wrote: post #4

Earlier quoted context omitted.

> lowest energy costs will likely be able to dictate market prices This is a good insight. I think everyone has seen that chart China's electricity generation going parabolic vs the US. That combined with cheaper yet equally good talent means at least in that segment, the closed labs won't catch up anytime soon

> the closed labs won't catch up anytime soon Which closed labs won’t catch up to whom?

I should have expanded, but basically, the OSS models becoming more and more capable to solve all day to day SWE coding needs will take a cut from frontier labs revenue.

Not to say that frontier labs won't make progress, but the bar for a sufficiently capable agent is all the OSS models need to meet to make this happen. I imagine a lot of hybrid setups where something like Opus is used only for planning/architecture, and anecdotally, the real token consuming part is implementation not architecture.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#15

Post by Aboutplants »

jqpabc123 wrote: post #2

The current closed source frontier models are more capable than the latest from DeepSeek. But is the capability difference enough to justify a 30x price difference? "Frontier models" are caught in a financial dilemma of their own making --- they have spent such huge sums on development and as a result, they may have inadvertently priced themselves out of the market. Energy costs are a huge factor for AI. He who has t…

I’ve been on this issue for a while now, models are not going to matter as much in the future. Pure energy cost will be the determining factor in who is most successful. The US just cannot build cheap energy the way other China can and at the scale that China will build it. 10 years from now it will be seen as the single source of advantage

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#16

Post by EGreg »

Hamuko wrote: post #9
EGreg wrote: post #6

Earlier quoted context omitted.

This sounds to me like the Bitcoin bros. Yes, the first-gen technology was very energy-heavy, but afterwards people (bitcoin maxis and people who held the bag) kept insisting that all new technology is “shitcoins” and that everyone should just buy bitcoin. Actually, platforms that serve many customers can bring down the costs tremendously through caching, and don’t need the AI credits as much: https://safebots.ai/cos…

Bitcoin is a poor analogue for much anything since it's very much designed to be energy-heavy.

Oh, and neural networks doing a huge number of floating point operations per word is not energy-heavy?

Training these neural networks every few months isn’t energy-heavy?

Both Bitcoin and these large models weren’t “designed to be energy-heavy”. It was a consequence of first-gen design decisions to solve a specific problem. Then as time went on, costs went down and they became a huge outlier in terms of energy. The question is whether the bagholders (the AI companies that invested untild amounts into the initial training) will fight to keep people using their tech and fearmonger about everything else.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#17

Post by throwfaraway4 »

treis wrote: post #12
jqpabc123 wrote: post #2

The current closed source frontier models are more capable than the latest from DeepSeek. But is the capability difference enough to justify a 30x price difference? "Frontier models" are caught in a financial dilemma of their own making --- they have spent such huge sums on development and as a result, they may have inadvertently priced themselves out of the market. Energy costs are a huge factor for AI. He who has t…

Historically the winners in software have a flywheel that turns faster with more users. Facebook the more of your friends on it the better the product was. Google tracked how long users were on pages to improve search. The frontier models are going to win that way. They won't feed your code back into the system but they will track which code you keep and what code gets a "try again claude". They're not going to lose…

>They're not going to lose on price. No consumer software ever has

Lists examples of software that are free to the users

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#18

Post by rgbrenner »

GodelNumbering wrote: post #4
jqpabc123 wrote: post #2

The current closed source frontier models are more capable than the latest from DeepSeek. But is the capability difference enough to justify a 30x price difference? "Frontier models" are caught in a financial dilemma of their own making --- they have spent such huge sums on development and as a result, they may have inadvertently priced themselves out of the market. Energy costs are a huge factor for AI. He who has t…

> lowest energy costs will likely be able to dictate market prices This is a good insight. I think everyone has seen that chart China's electricity generation going parabolic vs the US. That combined with cheaper yet equally good talent means at least in that segment, the closed labs won't catch up anytime soon

> China's electricity generation going parabolic

Even if we all switch to Chinese models, the west isn't going to be running the model on Chinese servers... and the majority of costs are from inference.

> cheaper yet equally good talent

China has tech talent, but this isn't a 3rd world developing nation. Chinese AI researchers are getting paid $10M+ USD/year salaries.

Also they're equally good, but somehow consistently behind?

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#19

Post by gentleman11 »

Aboutplants wrote: post #15
jqpabc123 wrote: post #2

The current closed source frontier models are more capable than the latest from DeepSeek. But is the capability difference enough to justify a 30x price difference? "Frontier models" are caught in a financial dilemma of their own making --- they have spent such huge sums on development and as a result, they may have inadvertently priced themselves out of the market. Energy costs are a huge factor for AI. He who has t…

I’ve been on this issue for a while now, models are not going to matter as much in the future. Pure energy cost will be the determining factor in who is most successful. The US just cannot build cheap energy the way other China can and at the scale that China will build it. 10 years from now it will be seen as the single source of advantage

You mean coal?
Post reply on HN