Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Outsourcing plus local AI will soon become more economical vs. frontier labs

signalbloom.ai

Post reply on HN
Print view View original

271–280 of 408 posts

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#271

Post by lmm »

goosejuice wrote: post #267
cdata wrote: post #261

Earlier quoted context omitted.

> There's enough evidence for the counter argument that this is essentially misinformation. > No evidence is shared Help an open-minded critic out.

Brand new industry, massive capital, dropping inference costs, increasing availability of compute, cost centers / subsidized subscriptions are common in SaaS, heavy competition, no public information on actual utilization rates. How much is Waymo burning a year? 3B on 300M ARR? Anthropic is what 5B on 20B ARR? Waymo is 3x older. Why don't we hear such confident statements about how subsidized their rides are? It's on…

> How much is Waymo burning a year? 3B on 300M ARR? Anthropic is what 5B on 20B ARR? Waymo is 3x older. Why don't we hear such confident statements about how subsidized their rides are?

We do. We hear it less often because no-one is talking about how Waymo changes how we all need to work or whatever, that's all.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#272

Post by lunar_mycroft »

goosejuice wrote: post #267
cdata wrote: post #261

Earlier quoted context omitted.

> There's enough evidence for the counter argument that this is essentially misinformation. > No evidence is shared Help an open-minded critic out.

Brand new industry, massive capital, dropping inference costs, increasing availability of compute, cost centers / subsidized subscriptions are common in SaaS, heavy competition, no public information on actual utilization rates. How much is Waymo burning a year? 3B on 300M ARR? Anthropic is what 5B on 20B ARR? Waymo is 3x older. Why don't we hear such confident statements about how subsidized their rides are? It's on…

Do people commonly argue Waymo isn't subsidizing rates?

Also, we do have some evidence for my position:

- We know that the consumer Claude plans provide _way_ more tokens than you could get if you were paying API prices. This is a huge part of why Anthropic's limits on other harnesses for subscription customers is such a big deal. So either their profit margin on API tokens is absurdly high, most consumer subscribers don't come anywhere near their rate limits, or they're losing money on the consumer subscriptions. - It appears that complains about people running into rate limits are common, which suggests the "consumers usually don't use much of their subscription" explanation is incorrect. - We also know that Anthropic has just become profitable, almost certainly driven mostly by enterprise customers. This rules out the "they make a very high profit margin on the API" explanation, since if that was the case they'd likely have been profitable much earlier.

Taken together, I think the case that their consumer subscriptions lose them money on net is pretty strong, even though their enterprise subscriptions (and API pricing) does make them a profit.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#273

Post by try-working »

Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

DeepSeek and Xiaomi are so cheap there's no need to get a plan. Just use the API.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#274

Post by curt15 »

Npovview wrote: post #119
runtime_terror wrote: post #108

Earlier quoted context omitted.

> The subscription token price is 10x-40x cheaper than API pricing This is a temporary phenomenon. Expect either drastic price increases or draconian throttling or both in the coming months. These companies are operating at huge loses and have hundreds of billions in liabilities and commitments. They need to turn on the money faucet sooner than later.

Even with increased prices, AI enables velocity both in development and bugs fixing. Would companies want that? If prices are biting the company, I think companies will route all development and bugs fixing requests through few superperfomer developers with complete knowledge of the different components within the company (they will be the Queen Bees holding the company on their head). The rest of the company will be…

> Even with increased prices, AI enables velocity both in development and bugs fixing.

What about human understanding of the codebase that's essential to any project's long term health? Even "superperformer developers" eventually leave the company.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#275

Post by linkregister »

henry2023 wrote: post #266
fg137 wrote: post #213

Earlier quoted context omitted.

"shady third party" If Claude hosted on AWS bedrock is not considered trustworthy, I have some bad news for you.

Anthropic illegally downloaded virtually all copyrighted material in the world to train their models. What makes you think they will have even a little consideration for your IP?

*tortiously downloaded

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#276

Post by piskov »

freediddy wrote: post #59

My friend is an exec at a US software company and they are preparing to lay off a few teams of programmers in their Eastern European locations and replacing them with a small number of US programmers + AI. He said they are much more productive and produce new features much faster.

How much time do you think it would pass before that guy comes back to reality and will lay off a bunch of agents? :-)

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#277

Post by protocolture »

Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

>When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing.

These are loss leaders that will not be maintained over the long term. Already we see moves to restrict their usage and redirect people back to API pricing.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#278

Post by devmor »

3form wrote: post #226
devmor wrote: post #128

Earlier quoted context omitted.

> it never will be. Giving strong “640k is enough for anyone” vibes here.

640k statement was absolute, this one is comparative. Cloud should have more compute and efficiency than local. I wouldn't be 100% sure, as I don't know what I might not be seeing, but still. Whether that comparative advantage will matter, though, is a completely different question.

Gotcha, I think I misunderstood the statement as saying today’s cloud-required will never be local-capable.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#279

Post by protocolture »

goosejuice wrote: post #107
treis wrote: post #28

I think this misses the forest for the trees. Working with ChatGPT is eerily similar to working with offshore Indian devs back in my enterprise days. Productive if guided explicitly but if let run wild there's lots of WTF moments. LLMs are likely to replace outsourced devs because your employees that know the context can use LLMs to do what offshore devs did before.

There are developers outside of your country that are talented, speak your language competently, and willing to work for less pay. There are plenty of reasons to believe that such devs will increase in numbers.

I am sure they exist but they are never the ones that I have to work with.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#280

Post by Our_Benefactors »

zozbot234 wrote: post #218
Our_Benefactors wrote: post #206

Earlier quoted context omitted.

It runs like shit though in terms of tokens/second and still has a reduced context window. Vs a single claude prompt can easily get into 300k tokens without breaking a sweat. I want local AI to be a thing but the hardware isn’t here yet, because the only options are a Mac Studio or DGX machines strapped together. RAM prices needs to crash before local AI has a chance at actually competing.

The more recent Chinese models are no longer heavily limited by context size. It can easily fit in RAM on a prosumer laptop. (You can also use swap space to extemd that, since context is only written to once per inference, thus a relatively mild wear-and-tear concern.)

Claude has 1M context window for the enterprise. 128k feels like a toy in comparison.
Post reply on HN