Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is minimal downside to switching to open models

marble.onl

Post reply on HN
Print view View original

131–140 of 351 posts

Re: There is minimal downside to switching to open models

#131

Post by linzhangrun »

sho wrote: post #96
linzhangrun wrote: post #16

Open source models are still not good enough for now, but with the current speed of one new SOTA every two months, by this time next year we will definitely have cheap open source models at least as good as Fable :)

I don't think we will. The open model labs are too resource constrained to approach Fable or even Opus on the general case and I don't see that changing within a year. Right now, due to profound shortfalls in both data and hardware compared to the US labs, the OSS models are IMO basically technology demonstrators that in practise are even more jagged than the US labs' efforts. The high points of the jaggedness are cl…

deepseek-v4-pro, probably the representative cheap opensouce LLM, was released in 2026.4 One year before, what OAI had in hand was gpt-4.1 and gpt-o3. I think it is not very controversial to say that deepseek is stronger than them, at most you can point to some post-training problems, basically the instability you mentioned. Also I am not sure if it is because the people who are best at using AI -- the people making AI -- get more development speed as the models get smarter, but my feeling is model progress is getting faster and faster. GPT-3.5 and GPT-4 were almost one year apart. The disadvantage from hardware limits and compute shortage is visible from the size of chinese models. glm-5.2, which is claimed to be around opus-4.6 level in coding, is only 744B. But Chinese engineers are obviously, how to put it, getting very effective results on "performance at the same size". And that is not even talking about the advantages from China's electricity, manpower, or even "national will" to compete against America. So saying it may take three years to catch up with a gap that is now only several months looks too pessimistic. ChatGPT itself was released only three and a half years ago, and today is already a completely different world.

Re: There is minimal downside to switching to open models

#132

Post by OccamsMirror »

nessex wrote: post #88
LPisGood wrote: post #73

Earlier quoted context omitted.

People talk about this a lot. What I have never seen is a discussion of methods they might employ to degrade the models. Let’s say I’m a bad faith LLM operator, and I want to degrade my model so the next release looks better and people want to switch to the more expensive one. How would I do that?

They would quantize the model. That'd make it cheaper to run, and have slightly worse output but it would still generate outputs with a similar feel, derived from a compressed version of the same knowledge base etc. They wouldn't even need to do this uniformly, quantized versions of the model could be routed only a subset of the requests. They could do this to nerf the old model, or more likely just to give themselve…

I have had the same experiences you've had with 4.6 and it was ever since they brought out 4.7. It's fairly obvious they're doing something like you've said here.

Re: There is minimal downside to switching to open models

#133

Post by nerdsniper »

himata4113 wrote: post #130
coffinbirth wrote: post #70

> Open models are served via various means, some by the companies that released them and some by third parties like OpenRouter. Unfortunately, both of these routes are dodgier in terms of privacy and data sharing, and I would not feel the same comfort sending API calls containing client or confidential data to them. That's why I'm using eurouter.ai with the following routing rule for all my requests: { "model": "glm-…

The part that gets me about anthropic red lines is "of Americans", okay so the rest of the civilized world is up for grabs then? It's okay to destabalize allies with sabotaged tests (in machine learning) and data exfiltration outside America? What gets me the most is that they claim that the model should follow the https://www.anthropic.com/constitution and they claim that it's embedded into the model. However, syste…

> The part that gets me about anthropic red lines is "of Americans", okay so the rest of the civilized world is up for grabs then? It's okay to destabalize allies with sabotaged tests (in machine learning) and data exfiltration outside America?

Regardless of Anthropic's "moral" position (inasmuch as a corporation can even have morals) against spying on non-Americans, they would have no way to enforce that limitation against the government because non-citizens outside of the USA have no protections from the intrusions of the US government.

Re: There is minimal downside to switching to open models

#134

Post by foo42 »

inigyou wrote: post #66
matheusmoreira wrote: post #35

Earlier quoted context omitted.

There's at least the possibility that they intentionally degrade the models as time passes. We can't really verify that we're getting what we're paying for all of the time. All the more reason to invest in local inference.

What if the new model is exactly as good as the last model on launch day but better than the last model was on the new model's launch day because it was degraded? Every single time?

Makes me think of [shepherd tones](Shepard tone - Wikipedia https://share.google/xooRbF7wIIhcsTt2J) which sounds like they're rising in pitch indefinitely

Re: There is minimal downside to switching to open models

#135

Post by delis-thumbs-7e »

cpursley wrote: post #94
earthnail wrote: post #90

Earlier quoted context omitted.

From all the large governmental institutions, the EU is the one currently holding up traditional western values. That gives it street cred in this subject.

With all the issues in the US and generally wrong direction, I can’t remember them ever arresting people for mean tweets in the way that Germany and the UK have. They all seem to be running full speed towards a surveillance state.

[x] Doesn’t know UK not in EU [x] Thinks people inciting violence online a free speech -issue [x] Calls Germany a surveillance state when US uses Palantir - a US company - to openly spy on its citizens

X seems to work great. Inciting men in with gambling, porn, crypto, ai and other broistan staples, then feeding them far-right nonsense info points.

Re: There is minimal downside to switching to open models

#136

Post by Phlogi »

throwaway27448 wrote: post #87
coffinbirth wrote: post #70

> Open models are served via various means, some by the companies that released them and some by third parties like OpenRouter. Unfortunately, both of these routes are dodgier in terms of privacy and data sharing, and I would not feel the same comfort sending API calls containing client or confidential data to them. That's why I'm using eurouter.ai with the following routing rule for all my requests: { "model": "glm-…

Why use EU specifically? I get not trusting the US, of course, but surely the EU isn't far behind in its desire to spy on its own citizens. Do you not live there?

US Data Privacy is not sufficient.

Re: There is minimal downside to switching to open models

#137

Post by zingar »

lmm wrote: post #127
leptons wrote: post #119

Earlier quoted context omitted.

>and US citizens don't seem to care I think maybe you haven't been paying attention. Most of us do care. Trump's approval rating is pretty low at 36%, and his disapproval rating is high. Just because he's still causing chaos doesn't mean the majority of us don't care about it. There's just no legal way to remove him, and his cronies simply won't do it - there's not enough votes in congress or he would have been gone…

This isn't about Trump. No 4th amendment rights at the border has been an issue for at least 20 years, but US citizens don't care because it doesn't affect citizens.

Yep, this was an issue long before Trump. They’ve just amped up the scale and stopped bothering with the deceit that they know doesn’t bother Americans.

Re: There is minimal downside to switching to open models

#138

Post by rob74 »

lysium wrote: post #124
leptons wrote: post #119

Earlier quoted context omitted.

>and US citizens don't seem to care I think maybe you haven't been paying attention. Most of us do care. Trump's approval rating is pretty low at 36%, and his disapproval rating is high. Just because he's still causing chaos doesn't mean the majority of us don't care about it. There's just no legal way to remove him, and his cronies simply won't do it - there's not enough votes in congress or he would have been gone…

I understand your point, however I don’t buy „there's just no legal way to remove him“. With so low ratings where are the daily protests against such type of government? Surely, nationwide daily protests would make elected officials reconsider their positions, given an upcoming midterm election, while there still is one. Don’t get me wrong, I know the thousands reasons why you won’t join a protest, I’m „guilty“ mysel…

Then again, nationwide daily protests would give the Trump administration an excuse to send ICE / the army / whoever else they can send to the cities where the protests take place (I guess they would be mostly blue-leaning ones) to "restore order", and at the same time lay the groundwork for influencing the November elections.

But the turnout at the periodic nationwide "No Kings" protests has been very good, and they have fortunately stayed peaceful.

Re: There is minimal downside to switching to open models

#139

Post by throwaway27448 »

Phlogi wrote: post #136
throwaway27448 wrote: post #87

Earlier quoted context omitted.

Why use EU specifically? I get not trusting the US, of course, but surely the EU isn't far behind in its desire to spy on its own citizens. Do you not live there?

US Data Privacy is not sufficient.

For what? Does the EU not want to spy on its citizens? That strikes me as... unlikely.

Why not host in east asia? Or southeast asia? Or south america? Or africa? Then you avoid both the government with incentive to spy on you (assuming you live in the EU) and american companies.

Re: There is minimal downside to switching to open models

#140

Post by Phlogi »

ttoinou wrote: post #102
coffinbirth wrote: post #70

> Open models are served via various means, some by the companies that released them and some by third parties like OpenRouter. Unfortunately, both of these routes are dodgier in terms of privacy and data sharing, and I would not feel the same comfort sending API calls containing client or confidential data to them. That's why I'm using eurouter.ai with the following routing rule for all my requests: { "model": "glm-…

You dont care about which exact provider it is using behind the hood ?

No, as long as they follow the requirements, especially the data privacy agreements. What would you? Price?
Post reply on HN