Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Outsourcing plus local AI will soon become more economical vs. frontier labs

signalbloom.ai

Post reply on HN
Print view View original

131–140 of 408 posts

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#131

Post by ofjcihen »

xcskier56 wrote: post #44
lumost wrote: post #34

Earlier quoted context omitted.

How many of those wtf moments are simply from not “being in the room when it happened?” Most enterprise software is riddled with wtf moments demanded as one compromise or another.

There's always wtf, why did we add this feature, but at least in my experience, once a week or so I run into something in this category. Me: "AI, please cleanup/refactor/improve this thing" AI: "Roger that! I deleted the file so now it's perfectly clean" ... insert W.T.F.

Yeah unfortunately you have to be careful with words like “clean up”.

I’ve had it assume I meant the folder multiple times :/

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#132

Post by Npovview »

throwfaraway4 wrote: post #17
treis wrote: post #12

Earlier quoted context omitted.

Historically the winners in software have a flywheel that turns faster with more users. Facebook the more of your friends on it the better the product was. Google tracked how long users were on pages to improve search. The frontier models are going to win that way. They won't feed your code back into the system but they will track which code you keep and what code gets a "try again claude". They're not going to lose…

>They're not going to lose on price. No consumer software ever has Lists examples of software that are free to the users

I want AI to go the way of Linux. I hope we see that future.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#133

Post by zuzululu »

I keep seeing this narrative involving Deepseek as an example of OSS LLMs but they are subsidizing a huge amount of tokens at cost and one can easily understand why they are doing it if one is not lazy and think critically.

It's still far too costly and not effective to use Local AI that can match what the frontier models can offer, especially when the inference hardware is being heavily restricted due to geopolitical risks. Claims about local LLMs somehow putting these frontier companies a run for their money I find especially doubtful in the long run.

Tokens are getting expensive because they are beginning to corner the market and will use that advantage to limit hardware distribution within and beyond the borders.

It's more likely that some workflows will see more local LLMs but those will never be the ones that require frontier model level or beat the price that a lighter smaller version of frontier model will offer to capture that tail end

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#136

Post by Npovview »

CuriouslyC wrote: post #27
rgbrenner wrote: post #18

Earlier quoted context omitted.

> China's electricity generation going parabolic Even if we all switch to Chinese models, the west isn't going to be running the model on Chinese servers... and the majority of costs are from inference. > cheaper yet equally good talent China has tech talent, but this isn't a 3rd world developing nation. Chinese AI researchers are getting paid $10M+ USD/year salaries. Also they're equally good, but somehow consistent…

Training models is as much art as science at this point. There's no gap in scientific acumen at Chinese labs, but the US has more real world experience in the art of training large models, and the US has the capital allocation lead.

Yes but when the Heads of CCP make something their target they chase it with all their might. Read the recent news of the fact that Chinese AI researchers can't leave China. China is now going after the Diamond industry of India.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#137

Post by sourcecodeplz »

zuzululu wrote: post #133

I keep seeing this narrative involving Deepseek as an example of OSS LLMs but they are subsidizing a huge amount of tokens at cost and one can easily understand why they are doing it if one is not lazy and think critically. It's still far too costly and not effective to use Local AI that can match what the frontier models can offer, especially when the inference hardware is being heavily restricted due to geopolitica…

Don't think so, from what i've heard deepseek isn't loosing money on inference.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#138

Post by henry2023 »

ghrl wrote: post #117
henry2023 wrote: post #85

Earlier quoted context omitted.

local models are 3 to 6 months behind SOTA models with the huge benefit of not needing to send all your IP to a shady third party. If inference cost comes down (as it has been for the last few years) you’ll be able to run today’s SOTA in your laptop by the end of the year.

I would say that is highly unlikely if by SOTA models you are not just referring to coding benchmarks but more general purpose ability and domain-specific knowledge. For example Kimi 2.6, which is comparable to Opus 4.6, is roughly 500+GB large, and I don't see how that would run on consumer hardware anytime soon. Besides, this is not just about the technical feasibility, but also economically not viable whatsoever.…

Because privacy has perceived value.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#139

Post by sourcecodeplz »

ghrl wrote: post #117
henry2023 wrote: post #85

Earlier quoted context omitted.

local models are 3 to 6 months behind SOTA models with the huge benefit of not needing to send all your IP to a shady third party. If inference cost comes down (as it has been for the last few years) you’ll be able to run today’s SOTA in your laptop by the end of the year.

I would say that is highly unlikely if by SOTA models you are not just referring to coding benchmarks but more general purpose ability and domain-specific knowledge. For example Kimi 2.6, which is comparable to Opus 4.6, is roughly 500+GB large, and I don't see how that would run on consumer hardware anytime soon. Besides, this is not just about the technical feasibility, but also economically not viable whatsoever.…

It runs right now on 512gb RAM Macs and PCs.
Post reply on HN