Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is minimal downside to switching to open models

marble.onl

Post reply on HN
Print view View original

71–80 of 351 posts

Re: There is minimal downside to switching to open models

#71

Post by iot_devs »

Aurornis wrote: post #37
julianlam wrote: post #6

I think it's interesting that people write off open weight models because they're "a few months behind" proprietary models. I know LLMs move at the speed of light (especially these past few quarters), but if Opus and GPT "a few months ago" were really like open weight models, then there's really no reason to not switch, especially for those who were using these models a few months ago. Your codebase didn't change, so…

> I think it's interesting that people write off open weight models because they're "a few months behind" proprietary models I experiment a lot with the open models and I’m getting tired of this trope. I’m not yet convinced that even the best open weight models are equal to Opus from “a few months” ago. I know what the benchmarks say. I had higher hopes. My real experience just doesn’t match the benchmarks. I also do…

I would love if you could make some examples

Re: There is minimal downside to switching to open models

#72

Post by iot_devs »

Aurornis wrote: post #36

The headline says one thing, then the article text says this: > I’m hoping it’s going to be minimal. I have multiple subscriptions and I pay per token to try out different LLM providers through OpenRouter. I also run open weight models locally. I just can’t agree yet . The models from Anthropic and OpenAI really are that much better than anything else. The open weight models must be universally benchmaxxed across the…

Do you have any example?

Re: There is minimal downside to switching to open models

#73

Post by LPisGood »

matheusmoreira wrote: post #35
kgeist wrote: post #11

Earlier quoted context omitted.

Every new proprietary model is "groundbreaking" and "look, it just solved task X that no other model could solve," only to be referred to as "that crappy previous-generation model" a month later. So yeah, I'm totally fine using Kimi-2.7, GLM-5.2 or Deepseek-v4. I think we've already hit the ceiling and most improvements now seem to be from harness improvements and slightly better RL to improve reasoning/tool calling.

There's at least the possibility that they intentionally degrade the models as time passes. We can't really verify that we're getting what we're paying for all of the time. All the more reason to invest in local inference.

People talk about this a lot. What I have never seen is a discussion of methods they might employ to degrade the models.

Let’s say I’m a bad faith LLM operator, and I want to degrade my model so the next release looks better and people want to switch to the more expensive one. How would I do that?

Re: There is minimal downside to switching to open models

#74

Post by CamperBob2 »

inigyou wrote: post #67
awakeasleep wrote: post #63

Earlier quoted context omitted.

Why would you buy and build everything before the low probability catastrophe strikes, though? You don’t get any benefit from switching early and you pay a big opportunity cost.

because as soon as it strikes computer hardware will be completely unavailable to buy?

Also, there's a nontrivial learning curve involved in running your own inference server, once you move past the casual-goofing-around-with-llama-server stage. If you care about not being a sharecropper on Sam's or Dario's plantation, you should consider learning the ropes. Even if you don't put these skills to immediate use in your day job.

I didn't appreciate this until I started down that road myself.

Re: There is minimal downside to switching to open models

#75

Post by Lapel2742 »

awakeasleep wrote: post #63
matheusmoreira wrote: post #43

Earlier quoted context omitted.

Current prices are insane but at this point I'm starting to feel like it's an existential issue. I'm not a US citizen. At any point the USA could come up with some arbitrary export controls. Not having a computer capable of running at least Qwen is starting to actually seem risky to me. At least it's going to be usable as a very high end gaming PC.

Why would you buy and build everything before the low probability catastrophe strikes, though? You don’t get any benefit from switching early and you pay a big opportunity cost.

> low probability catastrophe

There is also a low probability that someone enters peace negotiations solely to threaten the negotiators with death, yet here we are. With these guys it is: Better safe than sorry.

Re: There is minimal downside to switching to open models

#76

Post by [deleted] »

iot_devs wrote: post #72
Aurornis wrote: post #36

The headline says one thing, then the article text says this: > I’m hoping it’s going to be minimal. I have multiple subscriptions and I pay per token to try out different LLM providers through OpenRouter. I also run open weight models locally. I just can’t agree yet . The models from Anthropic and OpenAI really are that much better than anything else. The open weight models must be universally benchmaxxed across the…

Do you have any example?

[deleted]

Re: There is minimal downside to switching to open models

#77

Post by brucehoult »

marcus_holmes wrote: post #62
brucehoult wrote: post #38

Earlier quoted context omitted.

A Mac is cheaper than a high end GPU with the same amount of RAM.

ah, right, so it's about Apple Silicon being fast enough to use instead of a GPU?

They use the GPU but an Apple Silicon GPU has the same high speed access to all the RAM on the machine as the CPU does, rather than having its own walled-off maybe 16 GB VRAM in mainstream gaming GPUs or 24 GB in RTX 4090 or RTX 5090 (MSRP $1999 but in practice $3000-$4000 at the moment). Nvidia A100 (80GB VRAM) apparently cost $15,000 or so.

Not only does Apple's unified memory give the GPU more RAM to use, but it also eliminates copying things between CPU RAM and GPU RAM.

A Mac Mini with 48 GB RAM costs $1799. A Mac Studio with 96 GB RAM is $3999 — until March you could get a Mac Studio with 512 GB RAM for $3999, all of which could be used for your AI model.

https://www.tomshardware.com/tech-industry/apple-pulls-512-m...

Some are coming up used at silly prices.

https://www.trademe.co.nz/a/marketplace/computers/desktops/a...

NB NZ$44,999 is "only" US$25,772.

Re: There is minimal downside to switching to open models

#78

Post by whatever1 »

Claude started becoming useful for my coding purposes after it hit version 4.6. After that sure some nice to have additions but I think if I had 4.6 sonnet & opus as open weights, I would not need something more.

Having played a bit with Fable, reinforced the above.

Re: There is minimal downside to switching to open models

#79

Post by notatoad »

pkulak wrote: post #28

Sure. But OpenAI is the same price. Why would I pay $18/month for z.ai when OpenAI is $20/month?

the pricing page doesn't seem to call it out anymore, but the claim on z.ai coding plan used to be 3x the usage of the equivalent-price claude plan. whether that's accurate i don't know, but just based on api pricing GLM is way cheaper.

Re: There is minimal downside to switching to open models

#80

Post by OtomotO »

manyatoms wrote: post #60
matheusmoreira wrote: post #35

Earlier quoted context omitted.

There's at least the possibility that they intentionally degrade the models as time passes. We can't really verify that we're getting what we're paying for all of the time. All the more reason to invest in local inference.

Unless what you're getting is really explicitly spelled out in a contract, you should flatly assume that they're doing whatever they like whenever they like.

Even if it's in the contract, but can't be verified.
Post reply on HN