Why is it always the idea that the developers the work is outsourced to is less skilled than the local developers?
Outsourcing plus local AI will soon become more economical vs. frontier labs
331–340 of 408 posts
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#332I think this misses the forest for the trees. Working with ChatGPT is eerily similar to working with offshore Indian devs back in my enterprise days. Productive if guided explicitly but if let run wild there's lots of WTF moments. LLMs are likely to replace outsourced devs because your employees that know the context can use LLMs to do what offshore devs did before.
There are developers outside of your country that are talented, speak your language competently, and willing to work for less pay. There are plenty of reasons to believe that such devs will increase in numbers.
[dead]
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#333I think this misses the forest for the trees. Working with ChatGPT is eerily similar to working with offshore Indian devs back in my enterprise days. Productive if guided explicitly but if let run wild there's lots of WTF moments. LLMs are likely to replace outsourced devs because your employees that know the context can use LLMs to do what offshore devs did before.
"offshore Indian devs" are no slouches. They have access to the same GPT models and likely cost a tenth of the median US salary. Businesses are always looking to lower marginal cost. They will hire 1 software architect in US to write specs and 10 software developers in India to babysit 100 agents.
I'm sure there's some good ones but most are bad.
Not because all Indian devs are bad (this is of course absolutely not true), but most of the good ones are either no longer in India or working in India but for something more prestigious and interesting than an offshoring shop
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#334Earlier quoted context omitted.
"offshore Indian devs" are no slouches. They have access to the same GPT models and likely cost a tenth of the median US salary. Businesses are always looking to lower marginal cost. They will hire 1 software architect in US to write specs and 10 software developers in India to babysit 100 agents.
I'm sure there's some good ones but most are bad. Not because all Indian devs are bad (this is of course absolutely not true), but most of the good ones are either no longer in India or working in India but for something more prestigious and interesting than an offshoring shop
Which can be said of any country. Most of the good devs in US are working for the best US comapnies and not for small companies with less budgets.
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#335Why is it always the idea that the developers the work is outsourced to is less skilled than the local developers?
For me it is more that outsourced devs lack context and long term responsibility. Plus they are generally billing an hourly rate so their incentive structure is not directly aligned with the business that is outsourcing the work.
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#336Earlier quoted context omitted.
There are developers outside of your country that are talented, speak your language competently, and willing to work for less pay. There are plenty of reasons to believe that such devs will increase in numbers.
Is that your experience from having hired one, or are you speculating?
My last job was a mix of on shore and off, and we had about a 40% success rate on the offshore compared to roughly 85% onshore in terms of people working out, but also a fairly small sample size.
Latin American countries have become more popular for offshoring lately as you can get cheaper than US rates but still have the same or similar time zones.
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#337When discussing LLM pricing, people are missing the plot. The subscription token price is 10x-40x cheaper than API pricing. Your 90$ Claude subscriptions give you close to $1000 to $4000 in equivalent API token pricing. The second issue is that the quality of the model “operator” makes a massive difference in the outcomes. Highly skilled senior devs who know how to prompt and have high agency will outperform team peo…
The subscription plans are the "first hit is free" plans. They're not gonna last and don't build anything serious based on them.
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#338Earlier quoted context omitted.
> The subscription token price is 10x-40x cheaper than API pricing This is a temporary phenomenon. Expect either drastic price increases or draconian throttling or both in the coming months. These companies are operating at huge loses and have hundreds of billions in liabilities and commitments. They need to turn on the money faucet sooner than later.
Even with increased prices, AI enables velocity both in development and bugs fixing. Would companies want that? If prices are biting the company, I think companies will route all development and bugs fixing requests through few superperfomer developers with complete knowledge of the different components within the company (they will be the Queen Bees holding the company on their head). The rest of the company will be…
> AI enables velocity both in development and bugs fixing
Or so they say. You'll have to trust those vibes blindly, because double-checking these claims apparently makes you an anti-science luddite.
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#339Earlier quoted context omitted.
Anything over 150 seats means you need to pay at token rates plus the $20/user. My day job is operational (no coding at all) and I'm spending ~$300 a month on a few chats with Claude/Cowork a day over the course of a month.
We deployed OpenWebUI with the Claude API the other day for employees. Someone sent ten messages (which appeared to just be reasonable day-to-day work), and we paid $200 for it. There were 44M input tokens, 100k output tokens, no cache hits at all. OpenWebUI reports 3M tokens used, Claude reports 44M, and I have no idea where the rest of the tokens went. This was all on a brand new API key, installed directly to the…
No cache hits seems ominous, could this be an OpenWebUI issue? It also seems ominous that Anthropic models are basically nowhere on the OpenWebUI leaderboards.
I'm only doing a cursory search, but it seems OpenWebUI doesn't support Anthropic caching, and they don't intend to? Other providers handle caching automatically (apparently?) but caching has to be specifically managed by the client with Anthropic. If that's correct that OpenWebUI doesn't support it, it would really send your costs spiralling, because you're being billed for all the tokens in the entire multi-turn conversation on every turn:
https://github.com/open-webui/open-webui/issues/4887
I have no experience with OpenWebUI though (honestly, first time I've heard of it). Just trying to be helpful. If I'm completely incorrect then apologies in advance for sending you down the wrong path.
Re: Outsourcing plus local AI will soon become more economical vs. frontier labs
#340Earlier quoted context omitted.
This kind of things can certainly be run locally, even on a small mini-PC, like a NUC, or even on a laptop, with the weights stored on SSDs. Like I have said, the problem is not that they cannot be run, but that they may run more slowly than it is acceptable for a given application. Depending on the model, the speeds reported for inference with weights stored on SSDs vary from one token every few seconds to at most a…
When people say that you "can't do" something what they actually mean is that it's completely impractical (if not impossible).
Whether something is "impractical" depends on your expectations. High-latency unattended inference is definitely viable, even though it doesn't align much with what's being run in hyperscale datacenters.