Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is minimal downside to switching to open models

marble.onl

Post reply on HN
Print view View original

281–290 of 351 posts

Re: There is minimal downside to switching to open models

#281

Post by wongarsu »

pjc50 wrote: post #237
Aurornis wrote: post #61

Earlier quoted context omitted.

The reason you don't see more of this is because everyone does the math, realizes it's not a good deal, and then gives up on the idea. There's a post at the top of /r/localllama about this exact math right now: https://www.reddit.com/r/LocalLLaMA/comments/1ubrcwj/tokenom... TL;DR: Running GLM 5.2 is going to cost about $20K minimum, and that's going to be painfully slow compared to the cloud hosted versions. Even the…

.. conversely, all the cloud LLMs are being subsidized by their investors in addition to massive economies of scale.

economies of scale are enough to explain the entire price difference. Running 8 concurrent requests at 100 token/s on $100k hardware is a lot cheaper than running one concurrent request at 20 token/s on $20k hardware

Re: There is minimal downside to switching to open models

#282

Post by wat10000 »

CamperBob2 wrote: post #279
wat10000 wrote: post #267

Earlier quoted context omitted.

The Budapest Memorandum only requires going to the Security Council if nuclear weapons are involved. There's no required action at all for non-nuclear attacks. This isn't "weaseling over the exact wording," it's just the plain language of the memorandum. It really amazes me how much misinformation is out there about this thing. It only has six points, each one a single paragraph long. It's very quick and easy to read…

You can argue all day about the letter versus the spirit of the Budapest memorandum, but good luck getting any other countries to give up their nukes in the future. That's only one consequence of Trump's de-facto betrayal of Ukraine in support of his daddy figure in the Kremlin.

I have a really hard time accepting the idea that the spirit of the memorandum was that the signatories should actively defend Ukraine against non-nuclear attack, when it would have been so easy to write that explicitly.

I completely agree about no countries giving up their nukes in the future, but that's a consequence of the weak agreement, plus other actions like knocking over Iraq and Libya but not North Korea, tearing up the JCPOA with Iran, and... well, it seems like non-proliferation is mostly lip service in general.

Re: There is minimal downside to switching to open models

#283

Post by HarHarVeryFunny »

Animats wrote: post #125

OK, now what? Someone offers open models as a service? That's basically a time-sharing computing business - people at terminals sharing remote computing resources. If you buy your own H100 it will be idle while you're typing or reading or thinking. So sharing makes sense. But it doesn't have to be an "AI company". It's just a compute service. The companies that offer web hosting could get into this.

There are lots of companies providing open models as a service. DeepInfra and Fireworks AI for example. Even Amazon for that matter.

Re: There is minimal downside to switching to open models

#284

Post by shever73 »

c-b wrote: post #265

What's confusing to me is that there is no discussion about the actual downside experienced it's just theoretical.

There seemed to be no real discussion about anything! I was expecting more of a conclusion, but the article did not support the proposition in the headline.

Re: There is minimal downside to switching to open models

#285

Post by [deleted] »

NewsaHackO wrote: post #255
cbolton wrote: post #208

Earlier quoted context omitted.

They can include these limitations in a contract which can be enforced like any contract.

I don't think they can, at least if they are making an argument for why the Defense Production Act should not apply to them. Their original argument is that they will not help with anything that is unconstitutional, such as the unlawful spying on American citizens, without a warrant.

[deleted]

Re: There is minimal downside to switching to open models

#286

Post by nerdsniper »

jazzyjackson wrote: post #263
NewsaHackO wrote: post #255

Earlier quoted context omitted.

I don't think they can, at least if they are making an argument for why the Defense Production Act should not apply to them. Their original argument is that they will not help with anything that is unconstitutional, such as the unlawful spying on American citizens, without a warrant.

I don’t think Defense Production Act lets the government takeover what you produce, just that if you sell something you have to prioritize selling to the Feds. There is also precedent that code is speech and the government cannot compel speech (this came up during the debacle where FBI wanted a backdoor to unlock iPhones and Apple said no, we’re not building that)

It’s slightly nuanced, but during COVID-19 the DPA was applied to GM for production of ventilator machines. GM had not yet ever made any ventilator machines, and had only just one week earlier begun laying the groundwork to partner with Ventec Life Systems to look into retooling their automotive electronics factory in Kokomo, IN to produce ventilators.

I agree that the Apple case indicates that there’s a lot of uncertainty around this type of issue, at least post 1953 when title II of the DPA expired after Youngstown Sheet & Tube Co. v. Sawyer (1952)

Re: There is minimal downside to switching to open models

#287

Post by 0xbadcafebee »

itake wrote: post #209
hypfer wrote: post #197

Earlier quoted context omitted.

I'm sorry, but.. are you being serious? Yes. Yes. The only way one can write secure software is by always using the latest SOTA model. Anything else is inefficient and vulnerable. I hate this platform

https://news.ycombinator.com/item?id=46809708 Maybe you missed this article, but vercel found it quite annoying to teach AI about the latest updates in the React Framework. I think you’re confusing my point. I’m not saying that only SOTA models can write secure software, I’m saying that the models produced today will write software that’s considered insecure by 2034 standards, thus you would require to burn more toke…

Another solution might also be to stop reinventing the wheel every few years. New languages aren't producing better software. But people keep churning new languages out, and they become popular because humans have emotional attachment to inanimate things. If humans weren't so emotionally involved with the code, AI could happily produce C/C++ software indefinitely. (And if we could kick our dependence on the fucking browser for an application platform, we wouldn't need the horror that is the JavaScript ecosystem)

Re: There is minimal downside to switching to open models

#290

Post by trollbridge »

t-3 wrote: post #210
trollbridge wrote: post #204

Earlier quoted context omitted.

As was the civil rights era. The people who chose to be violent made it take longer, actually.

I don't think that's really accurate. Without the perceived threat from the militant factions, the peaceful protesters wouldn't necessarily have had political backing and support. Peaceful movements had been suppressed until that point, after all.

I think it is. Change happened because a majority of reasonable people wanted it to change.

Terrorist attacks, kidnappings, etc made that change take longer. What made MLK Jr so unique was that he carried a message of peace, not a message of war.

The militant factions never had any real power and would have never been close to powerful enough to overthrow the government, and if they’d been more successful, would have swayed the masses’ opinions in the wrong direction.

Post reply on HN