Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Outsourcing plus local AI will soon become more economical vs. frontier labs

signalbloom.ai

Post reply on HN
Print view View original

151–160 of 408 posts

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#151

Post by bitmasher9 »

ZeroCool2u wrote: post #150

A crucial factor tech industry folks tend to ignore is how much executives value predictable costs. Cloud migrations got away with this, but still had to argue fiercely, because 'the cloud' and its serverless tech had the potential to significantly decrease overall spend for unpredictable, bursty workloads. The usual counter-argument is the operational burden, but human capital is also a relatively fixed cost. A dedi…

Do you really want to buy the 3rd or 4th most intelligent AI?

There’s so much uncertainty, it seems like the safe option is to give everyone a Claude or OpenAI subscription/api key until the frontier isn’t changing every six months.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#152

Post by [deleted] »

bitmasher9 wrote: post #151
ZeroCool2u wrote: post #150

A crucial factor tech industry folks tend to ignore is how much executives value predictable costs. Cloud migrations got away with this, but still had to argue fiercely, because 'the cloud' and its serverless tech had the potential to significantly decrease overall spend for unpredictable, bursty workloads. The usual counter-argument is the operational burden, but human capital is also a relatively fixed cost. A dedi…

Do you really want to buy the 3rd or 4th most intelligent AI? There’s so much uncertainty, it seems like the safe option is to give everyone a Claude or OpenAI subscription/api key until the frontier isn’t changing every six months.

[deleted]

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#153

Post by ramesh31 »

stymaar wrote: post #106
layer8 wrote: post #104

Earlier quoted context omitted.

> What's your source for Opus being a 5T model? Probably Elon Musk: https://eu.36kr.com/en/p/3760679047267075

[flagged]

People can simultaeneously be reprehensible idiots while being a reliable expert on something they have personally invested billions of dollars into and operate at scale.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#154

Post by Forgeties79 »

anthonypasq wrote: post #143
Forgeties79 wrote: post #123

Earlier quoted context omitted.

Let’s see it first. And without omitting training/infrastructure costs at that. Until then my comment is still accurate.

its a private company, what exactly do you expect to 'see'?

Anthropic IPO's in less than 5 months and I guarantee you any company that officially is in the black will proudly shout it from the rooftops.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#155

Post by awkwardpotato »

ramesh31 wrote: post #153
stymaar wrote: post #106

Earlier quoted context omitted.

[flagged]

People can simultaeneously be reprehensible idiots while being a reliable expert on something they have personally invested billions of dollars into and operate at scale.

He's also invested billions of dollars in SpaceX and Tesla... which he regularly makes wild claims about that are untrue.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#156

Post by adrian_b »

stingraycharles wrote: post #77
Chyzwar wrote: post #48

When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…

Also, your local hardware is in no way capable of running the types of models that the cloud providers do, it’s just not economically feasible, and it never will be.

Depends on what you mean by "economically feasible".

Even very cheap mini-PCs and laptops can run any of the models run by cloud providers, albeit at a much lower speed (i.e. with the weights stored on SSDs).

Whether such a low speed is useful, depends on the application. For something like a coding assistant or bug scanning, an instant response is desirable, but certainly not necessary.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#157

Post by m_kos »

alexriddle wrote: post #91
simonw wrote: post #79

Earlier quoted context omitted.

I learned today that the Anthropic "Enterprise" plan - the one big companies use because they need governance features and audit logs and all of that jazz - is billed at API token rates (plus $20/seat/month). So large companies are getting billed a lot more than those discount subscription plans.

Anything over 150 seats means you need to pay at token rates plus the $20/user. My day job is operational (no coding at all) and I'm spending ~$300 a month on a few chats with Claude/Cowork a day over the course of a month.

$300 is my employer's monthly cap on Claude Enterprise. It lasts me at most a week of moderate use. I would much rather get Codex Pro and Claude Pro or Max, which would cost ≤ $200. For $300, one could also add Gemini Ultra to the mix so I could have all three review each other's code, etc.

Claude can be very good but enterprise pricing doesn't make sense to me.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#158

Post by ern_ave »

spprashant wrote: post #55
treis wrote: post #28

I think this misses the forest for the trees. Working with ChatGPT is eerily similar to working with offshore Indian devs back in my enterprise days. Productive if guided explicitly but if let run wild there's lots of WTF moments. LLMs are likely to replace outsourced devs because your employees that know the context can use LLMs to do what offshore devs did before.

"offshore Indian devs" are no slouches. They have access to the same GPT models and likely cost a tenth of the median US salary. Businesses are always looking to lower marginal cost. They will hire 1 software architect in US to write specs and 10 software developers in India to babysit 100 agents.

> "offshore Indian devs" are no slouches.

What evidence is there of the quality of Indian devs specifically?

One signal I'd expect to see, for example, would be success in programming competitions. Here's the list of winners of the IOI competition [1] - India has won 3 times.

Meanwhile, Turkey has won 4 times, Estonia has won 5 times, and Vietnam has won 22 times!

Why should we suspect that there are more or better developers in Indian than in any of the countries that has produced more winners??

[1] https://stats.ioinformatics.org/countries/?sort=medals_desc

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#160

Post by lowbloodsugar »

If IT is a cost center, then a company has likely already outsourced (and if it's called IT it probably is). If you are a software development company, that makes money from software, then a local team of SDEs using what-ever AI they want is a competetive advantage vs local team of SDEs trying to deal with an 11.5 hour gap to India. AI is coming for software developer jobs, and its coming for: a/ the low skill ones and b/ the high skill ones where turn-around and iteration matters. I've worked with great engineers in India, but the time difference was brutal for our fast moving business.
Post reply on HN