Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is minimal downside to switching to open models

marble.onl

Post reply on HN
Print view View original

241–250 of 351 posts

Re: There is minimal downside to switching to open models

#241

Post by 0xDEAFBEAD »

mopsi wrote: post #240
0xDEAFBEAD wrote: post #229

Earlier quoted context omitted.

I'm of the opinion that there is considerably more wailing about US government propaganda than actual US government propaganda. People who reference supposed US government propaganda rarely provide much in the way of concrete examples. Probably because there are legal restrictions on covert propaganda in the US: https://www.law.cornell.edu/wex/covert_propaganda To be clear, I'm happy to grant that: * The Pentagon won…

"UN Security Council action" is a broad term that can include deployment of international UN-led military forces, as in the Korean War: https://en.wikipedia.org/wiki/United_Nations_Command A few years prior to the Budapest Memorandum, the UN Security Council had authorized military action to liberate Kuwait. 42 countries participated in the coalition that drove Iraqi forces out of Kuwait: https://en.wikipedia.org/wik…

"Russia blocks Security Council action on Ukraine"

...

"A ‘no’ vote from any one of the five permanent members of the Council stops action on any measure put before it. The body’s permanent members are: China, France, Russian Federation, the United Kingdom, and the United States."

https://news.un.org/en/story/2022/02/1112802

(emphasis mine)

This is 101-level UN stuff. If Ukrainian diplomats were unaware that Russia can veto Security Council resolutions, that means they were totally incompetent.

It's also misleading to say the US "strong-armed" Ukraine out of its nukes... it was originally Ukraine's idea to abandon nukes, and they didn't have the control codes for the nukes on their territory anyways. The US attempted influence via carrots (financial assistance), not sticks ("strong-arming").

In any case, we did far more than just bring it up at the UN for discussion. See this map from a year or two ago: https://pbs.twimg.com/media/HKNCFWPbEAA7p5g?format=jpg&name=...

Mostly, in response to US generosity, Europeans just complained that the US should give even more. Your comment illustrates this perfectly--you speak as though the US only responded via UN diplomacy, completely neglecting over one hundred billion dollars the US sent in Ukraine aid, to a country which is not even a treaty ally of ours. When Biden was president, right after he saved Ukraine's butt in the initial invasion, public opinion of the US in Europe was barely even net-positive.

The real question is why Europeans spend so much time harassing the US for Ukraine funds, and so little time harassing tight-fisted countries which are actually in Europe like Ireland, Switzerland, Austria, Spain, etc. The answer: Europe has a transatlantic philosophy that the US brings the guns and the Europeans bring the scolding. As long as Ireland/Switzerland/Austria/Spain nod along with the scolding, they are doing their part, as far as Europe is concerned.

Re: There is minimal downside to switching to open models

#242

Post by bob1029 »

pkulak wrote: post #28

Sure. But OpenAI is the same price. Why would I pay $18/month for z.ai when OpenAI is $20/month?

Why pay a monthly fee when you can pay for exactly the # of tokens you actually consume?

The API rates are very affordable once you start to optimize for the fact that prepaid tokens seem to massively outperform other kinds of tokens.

I can often do with 1 million tokens what my peers have failed to do with 100 million. For me to spend more than $200/m in prepaid API tokens I'd have to pull a 007 work schedule.

Re: There is minimal downside to switching to open models

#243

Post by 542458 »

tumdum_ wrote: post #199

I find the attitude shown in this post very surprising. On the one hand, the post starts with a story of adopting Linux and other FOSS. The core of FOSS is giving its users the ability to understand and modify software they run. On the other hand, the rest of the post is about using a tool (LLM) that the author has no way to modify and no way to understand. Huge matrices of floats are at best comparable to compiled c…

I’d disagree wrt “modify”. There are all sorts of tools for modifying LLM weights (ie to remove refusals, remove layers or experts, merge models, finetune, and more) and a quick glance at huggingface or civit will show those in very active use.

I don’t think the hardware requirements are relevant. If a research lab publishes the code their particle collider runs under the GPL, that doesn’t make it not OSS even though they’re the only ones on the planet with the hardware to run it.

Re: There is minimal downside to switching to open models

#244

Post by conception »

jbverschoor wrote: post #30
kgeist wrote: post #11

Earlier quoted context omitted.

Every new proprietary model is "groundbreaking" and "look, it just solved task X that no other model could solve," only to be referred to as "that crappy previous-generation model" a month later. So yeah, I'm totally fine using Kimi-2.7, GLM-5.2 or Deepseek-v4. I think we've already hit the ceiling and most improvements now seem to be from harness improvements and slightly better RL to improve reasoning/tool calling.

Not only that, but to me it seems that after a week the intelligence is being downscaled or routed. Maybe because of lack of capacity

You can check https://marginlab.ai/trackers/codex/

It’s pretty good at catching when performance is degraded. It was for a week or so before Fable launched for instance, probably due to a/b testing or capacity as you noted.

Re: There is minimal downside to switching to open models

#245

Post by ch4s3 »

whatever1 wrote: post #78

Claude started becoming useful for my coding purposes after it hit version 4.6. After that sure some nice to have additions but I think if I had 4.6 sonnet & opus as open weights, I would not need something more. Having played a bit with Fable, reinforced the above.

I agree and I'd love for local models to hat the sonnet 4.6 level but nothing seems really all that close, and I'm not particularly excited about giving money to deepseek.

Re: There is minimal downside to switching to open models

#246

Post by laserlight »

kgeist wrote: post #11
julianlam wrote: post #6

I think it's interesting that people write off open weight models because they're "a few months behind" proprietary models. I know LLMs move at the speed of light (especially these past few quarters), but if Opus and GPT "a few months ago" were really like open weight models, then there's really no reason to not switch, especially for those who were using these models a few months ago. Your codebase didn't change, so…

Every new proprietary model is "groundbreaking" and "look, it just solved task X that no other model could solve," only to be referred to as "that crappy previous-generation model" a month later. So yeah, I'm totally fine using Kimi-2.7, GLM-5.2 or Deepseek-v4. I think we've already hit the ceiling and most improvements now seem to be from harness improvements and slightly better RL to improve reasoning/tool calling.

Don't forget the fact that you'll be questioned to death when you criticize the current generation of models, but somehow, when the new models arrive you'll be questioned to death if you don't find them better than the old ones.

Re: There is minimal downside to switching to open models

#247

Post by derwiki »

59nadir wrote: post #156
moomoo11 wrote: post #69

Earlier quoted context omitted.

ok but your competition using the latest models has an advantage not all of us are doing noob shit lol

Heh, if you're using LLMs heavily for work I think odds are pretty good you're doing pretty trivial stuff. It might not be trivial to you, but you're probably just not very good at this.

Pretty sure the big quant shops heavily use LLM; maybe it’s trivial stuff and they just work 100 hrs/week?

Re: There is minimal downside to switching to open models

#248

Post by aitchnyu »

flexagoon wrote: post #182
pkulak wrote: post #28

Sure. But OpenAI is the same price. Why would I pay $18/month for z.ai when OpenAI is $20/month?

OpenCode Go is $10/month and the limits are much more generous than those or Codex

After all the articles calculating OpenAI and Anthropic giving heavily subsidizing their subscriptions, how does OpenCode Go manage to be even cheaper?

Re: There is minimal downside to switching to open models

#249

Post by derwiki »

julianlam wrote: post #6

I think it's interesting that people write off open weight models because they're "a few months behind" proprietary models. I know LLMs move at the speed of light (especially these past few quarters), but if Opus and GPT "a few months ago" were really like open weight models, then there's really no reason to not switch, especially for those who were using these models a few months ago. Your codebase didn't change, so…

OOC did an LLM write this? The last sentence feels very LLM

Re: There is minimal downside to switching to open models

#250

Post by Aurornis »

pjc50 wrote: post #237
Aurornis wrote: post #61

Earlier quoted context omitted.

The reason you don't see more of this is because everyone does the math, realizes it's not a good deal, and then gives up on the idea. There's a post at the top of /r/localllama about this exact math right now: https://www.reddit.com/r/LocalLLaMA/comments/1ubrcwj/tokenom... TL;DR: Running GLM 5.2 is going to cost about $20K minimum, and that's going to be painfully slow compared to the cloud hosted versions. Even the…

.. conversely, all the cloud LLMs are being subsidized by their investors in addition to massive economies of scale.

It is false to say that all cloud LLMs are subsidized. The open weights models are hosted through numerous third party providers on OpenRouter that are operating as hosting businesses. They aren’t spending investor money to provide tokens for you at below-cost rates. They’re operating as hosting businesses.
Post reply on HN