Corporate America is getting hooked on open-source AI • Hacker News Board

Re: Corporate America is getting hooked on open-source AI

#101

Post by mylies43 » Fri, Sep 04, 2026, 5:47 PM UTC

woah wrote: post #85

biophysboy wrote: post #83

I have recently come to the conclusion that thinking for 2 seconds and using a cheap model with a slightly more detailed prompt works just as well as zero-shotting an idea with a fancy model. I work in science, and instead of asking the model “write a topic extraction algorithm”, I just say “hey look at this matrix factorization script I found in a repo, now make it use plotly and duckdb”. Have others come to the sam…

Really? It's worth it to you to spend 10 minutes thinking about how to prompt a dumber model to save $0.05? (not that open source models are dumber any more)

Well its def not $0.05, I just started using claude sonnet 5 and Ive found most simple questions might be 0.05 cents, a unit test is something like 0.10 -> 0.20 and small features and classes get into the individual dollars. Sure its a lot faster but at the end of the day its not cheap.

Plus there is something to say about being in the drivers seat, youll have a much better idea of how it works instead of needing to talk to claude and hope its correct. Since most LLMs also not very good at ideas even in my experience with better models its better to think for 10mins, youll get a much high quality result

Re: Corporate America is getting hooked on open-source AI

#102

Post by kittikitti » Fri, Sep 04, 2026, 5:49 PM UTC

Some of the most insidious parts of AI infrastructure includes the embedding model. Corporations have already spent an outstanding amount of time and money creating embedding vectors that are closed source and not reproducible. This means that all their data is locked into whatever embedding model they chose initially.

I highly recommend utilizing an open sourced embedding model instead of paying for a closed source one. It's vastly more reasonable to run an open sourced embedding model as a first step. They're much, much smaller and, due to the overhead of network latency, and running it locally has almost the same speed as through an API even on slow computers.

I would even go so far as to say that closed source embedding models have a high risk of data hostage. If a team doesn't have access to the embedding model, the embeddings become useless. A corporation like OpenAI could, say, hike the prices to that model by 1000x and everyone would have to pay up or forfeit any utility of the data.

I envision a future where open source embedding models are shipped with relevant technologies and implemented by currently under-utilized chips like NPU's. A startup developing cheap microprocessors that can run them is an idea I would pay cash for. Or perhaps they will be bundled with security tokens.

While it might be impractical for all corporate teams to run language models, it is very realistic for everyone to operate an open sourced embedding model, at least in their private cloud. Better yet, utilize transfer learning on an open sourced one to train your own, that way the embedding vector is more secure against competitors and trade secrets.

Re: Corporate America is getting hooked on open-source AI

#103

Post by Zambyte » Fri, Sep 04, 2026, 5:49 PM UTC

r_lee wrote: post #47

syntaxing wrote: post #22

I swear, Qwen 3.8 27B @ Q8 is smarter than Sonnet 5 most of the time. Why wouldn’t corporate America self host at this point, especially with better options like Deepseek Flash and GLM 5.3 flash that’s a middle ground between Sonnet and Opus

is this actually the case? I haven't kept up with the small models but if there's roughly Sonnet 4.6 level capable open small models, then I'd be impressed

Qwen 3.8 27b has the juice. Try it.

Re: Corporate America is getting hooked on open-source AI

#104

Post by cmiles8 » Fri, Sep 04, 2026, 5:50 PM UTC

intrasight wrote: post #89

theseamusjames wrote: post #82

Earlier quoted context omitted.

But they're all converging on capability. Do I care if it's a 72% or 74% on SWEBench? Practically, probably not. And if I'm not paying per token locally, then if it takes a tiny bit longer to get to the result, I don't care.

Smartphones and laptops are also "converging", but Apple is always a year or two ahead so it doesn't matter. "Converging" is a meaningless term when things move quickly and cost billions to develop.

Bad comparison.

1. People buy Apple because of the broader ecosystem of products and the “it just works” aspect of that ecosystem. Other companies make phones with features that are objectively better but folks don’t switch because the Apple ecosystem is sticky.

Despite trying, neither OpenAI nor Anthropic has managed to move up the stack beyond “hey guys new model release today!” announcements that everyone yawns at.

2. Switching costs are real. It’s a PITA to switch not just the phone but everything else. Switching model providers is a line of code and takes almost no effort.

Apple has a true moat which is why they can command a premium. OpenAI and Anthropic have no moat which is why they’re in trouble.

Re: Corporate America is getting hooked on open-source AI

#105

Post by transdev12 » Fri, Sep 04, 2026, 5:50 PM UTC

intrasight wrote: post #44

cmiles8 wrote: post #26

Every larger company I talk to these days has an active project on moving away from OpenAI and Anthropic to open models. And they’re actively shifting, as the article says, so the threat is far from theoretical. Unless they both dramatically slash prices then they’re in big trouble. Neither of them can afford to do that and both desperately need to convince the street that the opposite will happen if they want any ho…

There are most definitely is a moat - but it works both ways. The railguards in the models create moats keeping customers out. And the cost to build a modern agentic model is in the 10 figure range and growing. This is an expensive arms race that is going to create moats.

> the cost to build a modern agentic model is in the 10 figure range and growing

Source? The proliferation of labs building competent models would seem to suggest the opposite.

Re: Corporate America is getting hooked on open-source AI

#106

Post by Zambyte » Fri, Sep 04, 2026, 5:52 PM UTC

kittikitti wrote: post #102

Some of the most insidious parts of AI infrastructure includes the embedding model. Corporations have already spent an outstanding amount of time and money creating embedding vectors that are closed source and not reproducible. This means that all their data is locked into whatever embedding model they chose initially. I highly recommend utilizing an open sourced embedding model instead of paying for a closed source…

What are people even using embeddings for these days? It certainly seems like giving an agent grep covers most of the use cases. Dare I say: grep is all you need.

Re: Corporate America is getting hooked on open-source AI

#107

Post by GrinningFool » Fri, Sep 04, 2026, 5:53 PM UTC

simianwords wrote: post #50

cmiles8 wrote: post #26

Every larger company I talk to these days has an active project on moving away from OpenAI and Anthropic to open models. And they’re actively shifting, as the article says, so the threat is far from theoretical. Unless they both dramatically slash prices then they’re in big trouble. Neither of them can afford to do that and both desperately need to convince the street that the opposite will happen if they want any ho…

I know what happens in many big companies and not a single one is moving away from Anthropic/OpenAI/SpaceXAI. > Unless they both dramatically slash prices then they’re in big trouble False, they have already done so many times. > Neither of them can afford to do that and both desperately need to convince the street that the opposite will happen if they want any hope at a successful IPO. False, margins are higher and…

They kind of are fungible, up to a certain level of task. And much like most software developers don't need to exercise deep comp sci skills, most sw engineering doesn't have tasks that require the best models.

Re: Corporate America is getting hooked on open-source AI

#108

Post by transdev12 » Fri, Sep 04, 2026, 5:54 PM UTC

slowin wrote: post #3

I'd love to! For real coding though, SOTA models barely get the job done. It wasn't until Opus 4.5 that you could really get decent results. I'm sure this will change (and I can't wait for it!) but as of today, open models might be fine for summarizing and writing docs, but you need SOTA to work on code if you want to be competitive.

I generally use sonnet 5 for most coding tasks, a lot of coding is really pretty straightforward.

Re: Corporate America is getting hooked on open-source AI

#109

Post by overfeed » Fri, Sep 04, 2026, 5:54 PM UTC

gmadsen wrote: post #78

cmiles8 wrote: post #54

Earlier quoted context omitted.

But most commodities are the same way. It’s super expensive to drill for oil. I need oil and I’m in no position to mine my own because of the massive capital investment. But it doesn’t stop it from being a pure commodity. I couldn’t care less which company drilled for the oil… it’s all the same to me. Models are increasingly no different. OpenAI and Anthropic are a gas station saying “buy our gas for 10x the price!”…

The key difference is that the models upgrade multiple times a year. It is an inherently different than a commodity market

I don't think most motorists would care if OpenAI's gas stations just released 106-Octane "Intersteller" gas, unless their cars specifically require it.

Re: Corporate America is getting hooked on open-source AI

#110

Post by Zambyte » Fri, Sep 04, 2026, 5:55 PM UTC

hparadiz wrote: post #56

MrResearcher wrote: post #35

What IDE/extensions do you use for open-source LLMs? I tried VSCode with ollama and lm studio, and the experience is very subpar to the built-in copilot. It's not very usable.

https://huggingface.co/Qwen/Qwen3.8-27B No one I know uses the built in VSCode extensions anymore. It's all TUIs now. You can use Opencode as a TUI now for local.

Seconding the TUI usage. Opencode, codex, and pi are ones that I have / my friends have had success with using locally. I mostly use pi. It feels like the most "boring tool that just does its job" out of the big options.