There is minimal downside to switching to open models

Post by imhoguy » Mon, Jun 22, 2026, 10:42 AM UTC

johndough wrote: post #110

coffinbirth wrote: post #70

> Open models are served via various means, some by the companies that released them and some by third parties like OpenRouter. Unfortunately, both of these routes are dodgier in terms of privacy and data sharing, and I would not feel the same comfort sending API calls containing client or confidential data to them. That's why I'm using eurouter.ai with the following routing rule for all my requests: { "model": "glm-…

I had a look at eurouter.ai and it seems like an extremely bad offer. - The prices are ridiculous (15 % markup for free account). - They have a rate limit of 1000 requests per month, unless you pay 40€ per month for ... what exactly is their value proposition? - They have a single provider (TensorX) for DeepSeek-V4-Pro, with a cache read cost that is over 100 times higher than DeepSeek ($0.44 vs $0.003625). Notably,…

Post by trollbridge » Mon, Jun 22, 2026, 10:48 AM UTC

maybe_pablo wrote: post #84

LPisGood wrote: post #73

Earlier quoted context omitted.

People talk about this a lot. What I have never seen is a discussion of methods they might employ to degrade the models. Let’s say I’m a bad faith LLM operator, and I want to degrade my model so the next release looks better and people want to switch to the more expensive one. How would I do that?

Weight quantization, n-expert capping, routing to smaller model, context window truncation, aggressive sampling constraints, lossy speculative decoding and probably more.

Post by inigyou » Mon, Jun 22, 2026, 10:57 AM UTC

matheusmoreira wrote: post #101

CamperBob2 wrote: post #74

Earlier quoted context omitted.

Also, there's a nontrivial learning curve involved in running your own inference server, once you move past the casual-goofing-around-with-llama-server stage. If you care about not being a sharecropper on Sam's or Dario's plantation, you should consider learning the ropes. Even if you don't put these skills to immediate use in your day job. I didn't appreciate this until I started down that road myself.

> If you care about not being a sharecropper on Sam's or Dario's plantation Couldn't have put it better myself. That's what all this comes down to. Owning the hardware, owning the inference. Not perpetually renting them out on a meter like in the dystopian future they're envisioning.

Post by vrganj » Mon, Jun 22, 2026, 10:58 AM UTC

nxm wrote: post #159

solumunus wrote: post #104

Earlier quoted context omitted.

The UK can arrest you for hate speech. You can disagree with that policy on free speech terms if you want, and that’s really a maximal free speech position. It’s a very strange position to hold if you’re claiming that the U.S. is better when it comes to free speech. The U.S. administration is engaged in active smear campaigns against anyone who speaks loudly against them, threatened to revoke licenses of media compan…

I as a individual won’t get arrested for speaking my mind, and that’s much more important than some legal battle around corporate media. „ deployed the military in cities they don’t like for no other reason than intimidation of political rivals” That’s one perspective on simply trying to enforce laws. Moreover, let’s not forget about how Biden government tried to silence Rogan.

Post by 0xDEAFBEAD » Mon, Jun 22, 2026, 11:07 AM UTC

earthnail wrote: post #90

throwaway27448 wrote: post #87

Earlier quoted context omitted.

Why use EU specifically? I get not trusting the US, of course, but surely the EU isn't far behind in its desire to spy on its own citizens. Do you not live there?

From all the large governmental institutions, the EU is the one currently holding up traditional western values. That gives it street cred in this subject.