Live data from Hacker News

Outsourcing plus local AI will soon become more economical vs. frontier labs

signalbloom.ai

231–240 of 408 posts

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#231

Earlier quoted context omitted.

Their motives are to make the best product to compete in a very competitive market.

Right.. and their definition of a "best product" is theirs, not mine and probably not yours.

…the best model for agentic coding, is the top goal right now.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#232
"I phrase can this words a way not make sense. seems! but point across still!"

AI can turn it into a pseudo-poem or a 4 pages document. Or it can just fix the grammar. But it doesn't really change the point of the sentence– nor does it fix the actual issue with it.

Similar for code: There are codebases with lots of smells and really dirty parts, yet, that are still better than methodically clean ones that just don't "get to the point".

I am so sick of all the AI bloat. People were able to hide their incapability behind unnecessarily complex frameworks or obscuring it through "clean code" concepts. Now LLMs give those uninspired people the option to invest even less of what makes worthy software and hide it in more abstraction.

Just: AHA! (AI won't)

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#233
post #78

Earlier quoted context omitted.

What is expensive in nuclear energy? Reason there is not more of nuclear reactors is not the cost, it is regulation. Regulation can be changed (it also seem to already have, recently, IIRC - starting 2024 NRC law changes by Biden admin and later by Trump admin)

Reason there is not more of nuclear reactors is not the cost, it is regulation. The reason for regulation is that failure is not an option --- unless you're willing to accept the cost of making a big chunk of a state uninhabitable for a very long time. How much would failure cost? The Chernobyl exclusion zone is over 1000 square miles --- about half the size of Delaware. And it is expected to remain uninhabitable for…

[deleted]

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#234
post #205
post #79

Earlier quoted context omitted.

I learned today that the Anthropic "Enterprise" plan - the one big companies use because they need governance features and audit logs and all of that jazz - is billed at API token rates (plus $20/seat/month). So large companies are getting billed a lot more than those discount subscription plans.

> I learned today that the Anthropic "Enterprise" plan - the one big companies use because they need governance features and audit logs and all of that jazz - is billed at API token rates (plus $20/seat/month). Can large enterprises just not use the API ? I have audit logs and what seem to be enterprise features through my anthropic account (platform.claude.ai)

They can do that, but I expect they see individual user accounts and enterprise account management and easy rollout of Claude.ai/Cowork/Claude Code/etc as worth an extra $20/month/person.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#235
post #161

Earlier quoted context omitted.

It's 100% this. Many people suck at prompting. It's likely that habits from search are ingrained. But in general some people are just so bad at it .

According to Google, “there’s no wrong way to prompt”. https://www.youtube.com/watch?v=9bBfYX8X5aU&t=48s

No wrong way to [consume thing I sell that you'll consume more of if you do it poorly]

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#236

Earlier quoted context omitted.

But explain to me how these companies will recoup these costs outside of increasing inference pricing? Their business model is selling inference but the training and other costs have to be accounted for somehow. Unless I'm missing something obvious, inference costs must go up drastically if these companies are going to survive beyond the subsidy stage.

Sell more. The hope is that there is a huge addressable market that includes huge per-worker demand in almost all white collar work and lots of inference in people's private lives If that doesn't work, then yes, then prices will have to go up

Both anecdotally for myself and from what I'm reading in the news, it seems just as likely that AI usage has already largely peaked.

There was a lot of hype and exploration of capabilities, but models aren't evolving fast enough to keep that going, so I'm settling down into a familiarity with what an LLM can and can't do that means I am using them less overall that I was 6 months ago when I was throwing everything under the sun at it just to see what happened.

Without either new model breakthroughs or dramatically _lower_ costs, I will be very surprised if the ultimate market doesn't end up within an order of magnitude of where it is today.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#237
post #200

I've been saying this for a couple months now since I got decent hardware and started using my local Qwen 3.6 exclusively. I have no doubt the future for individuals and medium-sized companies is local private AI.

Could you share some of your hardware details for Qwen 3.6? And are you using the dense or MoE variant?

Sure, I have a 64G MBP with an M1 Ultra. The best model for me by far has been the 35B A3B, in particular the 8Q_KL unslouth variant. The dense model works but it's much slower, and I don't really see a difference in quality with a good harness.

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#238

I disagree with every part of this. Local LLMs are great and very useful but if you are claiming that their code quality is in the same ballpark as Claude Code or Codex with their best models I cannot consider you a serious person. I feel like this is analogous to the folks arguing that The Cloud is "someone else's computer." As if billions of dollars of spend gives these companies zero benefit over a Mac mini. Regar…

The cloud is in fact someone else's computer(s).

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#239
post #157

Earlier quoted context omitted.

$300 is my employer's monthly cap on Claude Enterprise. It lasts me at most a week of moderate use. I would much rather get Codex Pro and Claude Pro or Max, which would cost ≤ $200. For $300, one could also add Gemini Ultra to the mix so I could have all three review each other's code, etc. Claude can be very good but enterprise pricing doesn't make sense to me.

The $200 plan you're talking about is subsidized by Anthropic. They cannot afford to keep offering that to everyone indefinitely. Absolute best case scenario for current users is that they can continue to subsidize it as way to sell enterprise plans, but there's no way that they can keep offering it to everyone at those prices.

[dead]

Re: Outsourcing plus local AI will soon become more economical vs. frontier labs

#240
post #161

Earlier quoted context omitted.

It's 100% this. Many people suck at prompting. It's likely that habits from search are ingrained. But in general some people are just so bad at it .

According to Google, “there’s no wrong way to prompt”. https://www.youtube.com/watch?v=9bBfYX8X5aU&t=48s

Ehhh, their incentive in their marketing is to get normal people to not be intimidated by the big bad AI.

Power users are always going to have to take the messaging companies send out to the masses with a grain of salt.

Post reply on HN