Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Corporate America is getting hooked on open-source AI

nytimes.com

Post reply on HN
Print view View original

181–190 of 301 posts

Re: Corporate America is getting hooked on open-source AI

#181

Post by chvid »

novok wrote: post #174
cmiles8 wrote: post #26

Every larger company I talk to these days has an active project on moving away from OpenAI and Anthropic to open models. And they’re actively shifting, as the article says, so the threat is far from theoretical. Unless they both dramatically slash prices then they’re in big trouble. Neither of them can afford to do that and both desperately need to convince the street that the opposite will happen if they want any ho…

The open models are good because of distillation, which the US labs are actively working against via not revealing CoT ever and now you can see with OpenAI Astra 6 not even having a lot of CoT equivalents being emitted as tokens. Once the anti-distillation stuff is in place the open distillation models will probably start having larger and larger gaps. If the companies survive the next few years, which they probably…

Look up how many Chinese are pursuing a computer science degree.

Chinese AI labs do well because they have the best AI people coming out of a huge talent pool.

Re: Corporate America is getting hooked on open-source AI

#182

Post by CodingJeebus »

schopra909 wrote: post #51

IMO this will be a blip. There’s a lot of talk in the wake of all the Uber handwringing about token spend. Legacy enterprises want to look innovative to Wall Street without spooking them, so it’s easy to hop on the narrative and “show” that they’re innovating in a cost responsible manner. This feels reminiscent of the big push to RAG a few years ago. And, more broadly the skunkworks projects that big companies tout i…

I don't see this being a blip, for multiple reasons.

A) the uptime matrix between Github, OpenAI, and Anthropic means that we've faced multiple entire days of not being able to ship org-wide due to our reliance on automated code review and other tooling. Every cloud service baked into our CI pipeline becomes a point of failure. We can't live without AI anymore, but it too often either directly or indirectly gets interferes with our ability to ship, and I don't see this improving any time soon.

B) Locally hosted AI has serious advantages with regard to PII/sensitive data management, and there's not much the frontier models can do to overcome this. There are so many things I want to build and let loose in a sensitive data environment but can't due to data governance around frontier models.

C) Anthropic and OpenAI cannot keep prices low forever. They're still burning insane amounts of cash and at some point, they're going to have to transition from growth mode to profit mode. They're already juicing their sales pipelines to the max with introductory pricing and other things to get people in the door. But those are all short-term online marketing plays.

Re: Corporate America is getting hooked on open-source AI

#183

Post by Karrot_Kream »

piva00 wrote: post #169
Karrot_Kream wrote: post #165

Earlier quoted context omitted.

ChatGPT or ChatGPT Work or Codex? Are folks at your company still just using a chatbot to do work? (If so, this is quite surprising, as I find harness-based agent usage to be much much better than chatbotbot agent usage.)

No, the full suite: ChatGPT Work, Codex, chat, Deep Research, etc., same for Claude with Code, Cowork, etc. And any new release of model or tool is trialed and assessed. Just too many products to exhaustively list, it's a big tech company, we're trialing a lot under the sun to find workflows, tools, integrations, including a lot of bespoke internal research, there's a large ML department since almost the inception of…

Makes sense. If you have the resources, trialing out various models to find the pareto optimal point is worth it.

Re: Corporate America is getting hooked on open-source AI

#184

Post by saghm »

simianwords wrote: post #86
saghm wrote: post #84

Earlier quoted context omitted.

> I can have a formal bet that prices can go lower Is this a typo? Have you actually made a formal bet on a prediction market or something to put your money where your mouth is, or are you just saying that you could ? There's a lot of things I could plausibly make bets on, but that doesn't mean that they're likely to happen.

I can have a formal bet with anyone. No prediction market will ever have a stupid bet like this because anyone who has skin in game has studied it enough to know it is a 99.99% chance

Okay, I "can have" a formal bet with my friend that the sky will be green tomorrow, and I'm sure they'd agree to bet against that. I don't see why you think that's a useful statement in the context of arguing that it's a likely outcome though.

Re: Corporate America is getting hooked on open-source AI

#185

Post by overfeed »

woah wrote: post #85
biophysboy wrote: post #83

I have recently come to the conclusion that thinking for 2 seconds and using a cheap model with a slightly more detailed prompt works just as well as zero-shotting an idea with a fancy model. I work in science, and instead of asking the model “write a topic extraction algorithm”, I just say “hey look at this matrix factorization script I found in a repo, now make it use plotly and duckdb”. Have others come to the sam…

Really? It's worth it to you to spend 10 minutes thinking about how to prompt a dumber model to save $0.05? (not that open source models are dumber any more)

Its not just about saving $0.05. There are many legitimate reasons to not want the AI to do the thinking (aka architecting or planning) for you. In those cases where one-shotting is undesirable or unnecessary, closed frontier AIaaS has no advantages over open weights.

Re: Corporate America is getting hooked on open-source AI

#186

Post by Gareth321 »

cmiles8 wrote: post #26

Every larger company I talk to these days has an active project on moving away from OpenAI and Anthropic to open models. And they’re actively shifting, as the article says, so the threat is far from theoretical. Unless they both dramatically slash prices then they’re in big trouble. Neither of them can afford to do that and both desperately need to convince the street that the opposite will happen if they want any ho…

> However the cold reality for both is that there is zero moat to a model anymore.

The moat right now is a) the hardware, b) the electricity, c) the intelligence, and d) scalability.

On hardware, it's very expensive to purchase anything which can provide a fraction of the performance of a subscription. Traditional accounting depreciation would imply that purchasing local hardware is a terrible financial decision.

On electricity, this is a surprising cost center depending on location. A system with just one 5090 can easily pull 1kW, and to achieve usable performance for a workplace is going to require dozens of machines. This can represent an extra $10-20k in electricity in cheap places. In California or Europe this could be $30-60k per year.

As for intelligence, the frontier models from OpenAI and Anthropic are still superior, and they have at least a 3-6 month head start. Distilled models are closing the gap on some metrics, but they still can't compete. That's why they cost so much less.

The last major moat is the ability for subscriptions to scale with need. This means easily adding and removing licenses. This is far easier than purchasing extremely expensive hardware (and managing it), and selling it if/when internal demand changes. It's the same reason companies use contractors. The ramp up/down costs are very high.

The only real moat that local LLMs have right now is privacy.

Re: Corporate America is getting hooked on open-source AI

#187

Post by mlboss »

chvid wrote: post #181
novok wrote: post #174

Earlier quoted context omitted.

The open models are good because of distillation, which the US labs are actively working against via not revealing CoT ever and now you can see with OpenAI Astra 6 not even having a lot of CoT equivalents being emitted as tokens. Once the anti-distillation stuff is in place the open distillation models will probably start having larger and larger gaps. If the companies survive the next few years, which they probably…

Look up how many Chinese are pursuing a computer science degree. Chinese AI labs do well because they have the best AI people coming out of a huge talent pool.

In both the countries the researchers are Chinese.

Re: Corporate America is getting hooked on open-source AI

#188

Post by nvme0n1p1 »

novok wrote: post #174
cmiles8 wrote: post #26

Every larger company I talk to these days has an active project on moving away from OpenAI and Anthropic to open models. And they’re actively shifting, as the article says, so the threat is far from theoretical. Unless they both dramatically slash prices then they’re in big trouble. Neither of them can afford to do that and both desperately need to convince the street that the opposite will happen if they want any ho…

The open models are good because of distillation, which the US labs are actively working against via not revealing CoT ever and now you can see with OpenAI Astra 6 not even having a lot of CoT equivalents being emitted as tokens. Once the anti-distillation stuff is in place the open distillation models will probably start having larger and larger gaps. If the companies survive the next few years, which they probably…

If we accept the premise that the top Chinese labs are simply distilling and can't compete otherwise: why don't US labs simply do the same thing? Distill their own models and slash their costs by 99% while keeping the same quality output. It should be a piece of cake if even the open labs can figure it out, after all.

One way or another they're getting the same results as proprietary labs, with a fraction of the hardware for a fraction of the cost. OpenAI can't keep raising funding rounds of $100billions to subsidize their compute costs and get results by brute forcing parameter count. And if they're having trouble keeping up with Chinese labs' efficiency, maybe they should stop worrying about distilling and instead hire some of the smart people responsible.

Re: Corporate America is getting hooked on open-source AI

#189

Post by Gareth321 »

chvid wrote: post #181
novok wrote: post #174

Earlier quoted context omitted.

The open models are good because of distillation, which the US labs are actively working against via not revealing CoT ever and now you can see with OpenAI Astra 6 not even having a lot of CoT equivalents being emitted as tokens. Once the anti-distillation stuff is in place the open distillation models will probably start having larger and larger gaps. If the companies survive the next few years, which they probably…

Look up how many Chinese are pursuing a computer science degree. Chinese AI labs do well because they have the best AI people coming out of a huge talent pool.

And yet their models are all behind the US models. I don't think LLM progress is strictly correlated with the number of computer science degree holders in a specific country.

[There are many examples of distillation attacks.](https://cyberpress.org/anthropic-claude-ai-distillation-atta...) Of course, you could argue this is just a form of "learning" from other intelligent systems, and this is arguably what LLMs have been doing from the beginning. So I don't begrudge the Chinese labs doing it - they all do it. But let us not pretend that it's not happening.

Re: Corporate America is getting hooked on open-source AI

#190

Post by cousinbryce »

Kuyawa wrote: post #142
majorchord wrote: post #38

Earlier quoted context omitted.

Agreed. And conversely, American models can also just as easily be secretly influenced for bad things, or be more tightly controlled by the government, to corporate America's own detriment.

America's own demise will be made in America, stamped by American laws

From whence shall we expect the approach of danger? Shall some trans-Atlantic military giant step the earth and crush us at a blow? Never. All the armies of Europe and Asia...could not by force take a drink from the Ohio River or make a track on the Blue Ridge in the trial of a thousand years. No, if destruction be our lot we must ourselves be its author and finisher. As a nation of free men we will live forever or die by suicide. ― Abraham Lincoln
Post reply on HN