Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is minimal downside to switching to open models

marble.onl

Post reply on HN
Print view View original

271–280 of 351 posts

Re: There is minimal downside to switching to open models

#271

Post by fg137 »

helloplanets wrote: post #228
fg137 wrote: post #224

Earlier quoted context omitted.

If your only concern is data residency, data privacy and sharing, why not just use bedrock with the processing region locked to eu-west-2? For sure, it's not an European company serving the LLM, but it satisfies your requirements otherwise and is trusted by tons of companies worldwide.

Anthropic already explicitly communicated that they'll store and check all the data from Bedrock or any platform, even if you've selected zero data retention, if using Mythos class models. To use these models on any platform, you'll have to accept these terms regardless of the region. > Limited data retention and review as part of our safety work. Prompts submitted to, and outputs generated by, Mythos-class models ar…

The original comment is about GLM/deepseek models. As you already pointed out, this applies if you use those specific Claude models on any platform, so I don't know what the point is.

Re: There is minimal downside to switching to open models

#272

Post by WhyNotHugo »

coffinbirth wrote: post #70

> Open models are served via various means, some by the companies that released them and some by third parties like OpenRouter. Unfortunately, both of these routes are dodgier in terms of privacy and data sharing, and I would not feel the same comfort sending API calls containing client or confidential data to them. That's why I'm using eurouter.ai with the following routing rule for all my requests: { "model": "glm-…

You only need to worry about GDPR and the hoster being in the EU if you're giving the model actual access to production data — which you shouldn't anyway. Use the model to write code that processes or analyses the data, so that process can easily be reproduced with deterministic results.

Re: There is minimal downside to switching to open models

#273

Post by fg137 »

quikoa wrote: post #226
fg137 wrote: post #224

Earlier quoted context omitted.

If your only concern is data residency, data privacy and sharing, why not just use bedrock with the processing region locked to eu-west-2? For sure, it's not an European company serving the LLM, but it satisfies your requirements otherwise and is trusted by tons of companies worldwide.

> it's not an European company serving the LLM That's a pretty big downside if data privacy and sharing is one of the main concerns.

I'd like to see some real reasoning here that is based on facts.

Re: There is minimal downside to switching to open models

#274

Post by GL26 »

What makes an open model worse is ultimately the budget : you have access to worse data, not SOTA models, less GPU compute time, and having a good fine tuning team is extremely expensive. Linux works because the entry barriers are purely on a software side : a lot of contributers all around the world can outclass any OS by contributing on their scale to Linux. All you need to contribute is a computer, and your brain. Open models don't have the same community push, they rely on core ressources that not anyone owns. And injecting them in the model costs too much money. If there are no public breakthroughs in the way we train large open models that makes community led models 10x better, the shift to open models will never happen on a large scale.

Re: There is minimal downside to switching to open models

#275

Post by 0xbadcafebee »

pbgcp2026 wrote: post #167
0xbadcafebee wrote: post #111

Earlier quoted context omitted.

One reason might be request limits. OpenAI's ChatGPT Plus w/Codex ($20/month) provides a worst-case 5-hour-request-limit of 15 for GPT-5.5, 20 for GPT-5.4, 60 for GPT-5.4-Mini. Whereas Z.ai Lite ($18/month) provides a worst-case of ~80 for GLM 5.2 (off-peak; on-peak is 2am-6am New York time). So Z.ai can provide higher limits for a cheaper price. ( https://codeberg.org/mutablecc/calculate-ai-cost/src/branch/... )

Subscriptions are done . By the end of 2026 everyone will be paying for actual mils of tokens consumed, via API calls.

I don't see any indicator of that happening. And actually token count pricing is frequently being replaced with "credits pricing", and subscriptions with obscure variable limits

Re: There is minimal downside to switching to open models

#276

Post by CamperBob2 »

inigyou wrote: post #215
matheusmoreira wrote: post #101

Earlier quoted context omitted.

> If you care about not being a sharecropper on Sam's or Dario's plantation Couldn't have put it better myself. That's what all this comes down to. Owning the hardware, owning the inference. Not perpetually renting them out on a meter like in the dystopian future they're envisioning.

You also have the option to not use AI

Not in the future, not if you want to get paid.

Re: There is minimal downside to switching to open models

#277

Post by 0xbadcafebee »

aitchnyu wrote: post #248
flexagoon wrote: post #182

Earlier quoted context omitted.

OpenCode Go is $10/month and the limits are much more generous than those or Codex

After all the articles calculating OpenAI and Anthropic giving heavily subsidizing their subscriptions, how does OpenCode Go manage to be even cheaper?

OpenAI and Anthropic are trying to pay off a half trillion dollars of investment. They also have the most demand right now, to the point that Anthropic sometimes doesn't have enough compute and that means more limits. They can't stop taking new customers, though, because the market would hate it.

An open weight inference provider only needs to pay for GPUs, or discounted APIs from 3rd party vendors. Same basic financial model but they didn't spend a trillion dollars so their loss isn't as high so they can afford to do more inference for less money, and their demand isn't as high so there's more than enough compute.

Re: There is minimal downside to switching to open models

#278

Post by nerdsniper »

cbolton wrote: post #208
nerdsniper wrote: post #133

Earlier quoted context omitted.

> The part that gets me about anthropic red lines is "of Americans", okay so the rest of the civilized world is up for grabs then? It's okay to destabalize allies with sabotaged tests (in machine learning) and data exfiltration outside America? Regardless of Anthropic's "moral" position (inasmuch as a corporation can even have morals) against spying on non-Americans, they would have no way to enforce that limitation…

They can include these limitations in a contract which can be enforced like any contract.

FISA Section 702 (50 U.S.C. § 1881a) or CLOUD Act could be used to override any contractual terms that US government agencies may have agreed to. Those clauses would be unenforceable / unexecutable.

More generally it would be overpowered by the Sovereign Acts Doctrine.

The facts aren’t identical to the 2008 Yahoo FISCR case but that case sets the tone for how any clauses like this would just be brushed under the rug.

Re: There is minimal downside to switching to open models

#279

Post by CamperBob2 »

wat10000 wrote: post #267
mopsi wrote: post #240

Earlier quoted context omitted.

"UN Security Council action" is a broad term that can include deployment of international UN-led military forces, as in the Korean War: https://en.wikipedia.org/wiki/United_Nations_Command A few years prior to the Budapest Memorandum, the UN Security Council had authorized military action to liberate Kuwait. 42 countries participated in the coalition that drove Iraqi forces out of Kuwait: https://en.wikipedia.org/wik…

The Budapest Memorandum only requires going to the Security Council if nuclear weapons are involved. There's no required action at all for non-nuclear attacks. This isn't "weaseling over the exact wording," it's just the plain language of the memorandum. It really amazes me how much misinformation is out there about this thing. It only has six points, each one a single paragraph long. It's very quick and easy to read…

You can argue all day about the letter versus the spirit of the Budapest memorandum, but good luck getting any other countries to give up their nukes in the future.

That's only one consequence of Trump's de-facto betrayal of Ukraine in support of his daddy figure in the Kremlin.

Re: There is minimal downside to switching to open models

#280

Post by wongarsu »

Aurornis wrote: post #61
bnj wrote: post #41

I’ve been wanting to get better acquainted with local inference but I don’t have the hardware, which has made me think about something I haven’t seen discussed, which is local collaboratives. The economics makes it seem like a group of people joining together to run good hardware and an open model might make sense, but I haven’t seen anything like this mentioned. Have I been missing it? I think it would be pretty nea…

The reason you don't see more of this is because everyone does the math, realizes it's not a good deal, and then gives up on the idea. There's a post at the top of /r/localllama about this exact math right now: https://www.reddit.com/r/LocalLLaMA/comments/1ubrcwj/tokenom... TL;DR: Running GLM 5.2 is going to cost about $20K minimum, and that's going to be painfully slow compared to the cloud hosted versions. Even the…

If you invest the minimum to run the model, obviously that's more expensive per-token than investing the optimum to get the best price/performance tradeoff (which for GLM 5.2 is at least five times that figure)

If you can bring the load to run the model on close to optimal hardware 24/7 with multiple concurrent requests, and have reasonably cheap power and AC, you would break even in a reasonable timespan. Which won't happen unless you are self-hosting for a medium-sized company. I guess you could sell your spare capacity to get better utilization ... and we've reinvented hosted inference

Post reply on HN