251–260 of 291 posts
Earlier quoted context omitted.
Corporate America has been using open source for decades, and it wasn’t anywhere as slow as you portray. This argument simply doesn’t hold water. Besides, in their present rather dire financial state there isn’t much to sue these companies for anyway cash wise. NYTimes is suing on IP grounds.
it holds all the water, to this very day using open source in my client's projects requires approval from legal.
Earlier quoted context omitted.
I support and use open models as much as possible, but I'm not totally convinced that OAI or Anthropic have no moat, even as open models catch up to the frontier. Serving and inference are still hard problems when you're talking about a 2 trillion parameter model. Fine-tuning, if that remains a realistic need for businesses, is also a difficult infra problem at that scale. In the most bearish case, where there is no…
OK, but somehow there won't be companies who will sell you appropriate hardware and a turnkey system to serve inference? Or companies that will help you fine tune popular models? It's not about money, it's about control. Companies have lots of money and want control over their key technology.
The same dynamics that define the public cloud ecosystem are at play here. What AWS sells you is access to appropriate hardware and turnkey infra for your needs. Looking at the cloud industry over the last 20 years, I find it hard to believe that it is impossible to build a moat or a huge business around this.
Earlier quoted context omitted.
If companies are really doing this, then we're saying they have no problems spending tens of millions to get somewhat decent TPS and then having their employees complain they are timesliced and getting lots of timeouts because their org has 500 employees?
At least accord to the MIT study last year, most employees are using their own AI subscriptions to do work. Keep in mind that a vanishingly small number of workers are SWE's churning millions of tokens daily.
Earlier quoted context omitted.
Also people are people and they will get emotionally attached to claude :)
Or ready to get a divorce... The load-bearing seam of the relationship is affection versus annoyance.
Every larger company I talk to these days has an active project on moving away from OpenAI and Anthropic to open models. And they’re actively shifting, as the article says, so the threat is far from theoretical. Unless they both dramatically slash prices then they’re in big trouble. Neither of them can afford to do that and both desperately need to convince the street that the opposite will happen if they want any ho…
> However the cold reality for both is that there is zero moat to a model anymore. The moat right now is a) the hardware, b) the electricity, c) the intelligence, and d) scalability. On hardware, it's very expensive to purchase anything which can provide a fraction of the performance of a subscription. Traditional accounting depreciation would imply that purchasing local hardware is a terrible financial decision. On…
One of the basic questions/concerns here though is that it's not like the AI places are getting the GPUs for 10x less. It's true they have some economies of scale, but they also have some waste, and frankly in this particular case it's not clear they get that much gain over what a lot of businesses could achieve. The biggest traditional gain for central providers is that a lot of typical computing usage is burst-y, and in turn local kit might be underutilized. But with LLMs heavy users tend to use them all the time assuming their tokens allow it (and in the case of local hardware there's nothing stopping you, quite the contrary), they can use it directly interactively or leave them to go overnight on something too.
So it's reasonable to suspect that the reason subscriptions are only a fraction of the cost is that we're in a bubble seeing these companies losing money in an attempt to gain some sort of durable advantage. Just as every previous time, there is the chance that the music stops at some point, and they need to crank up pricing or pull other schemes to actually make money. Of course, it can be a good deal in the mean time, you basically get to suck down investor money for nothing, but it's also not unreasonable to at least be consider fallbacks. Even beyond questions of control and risk etc. I know at least a few places that are now genuinely considering questions like "what happens if a datacenter we depend on gets droned" that would have never had an iota of thought devoted to them even 5 years ago.
>On electricity, this is a surprising cost center depending on location. A system with just one 5090 can easily pull 1kW, and to achieve usable performance for a workplace is going to require dozens of machines. This can represent an extra $10-20k in electricity in cheap places.
I don't think that's "surprising" at all, everyone knows about power use. And this seems like it gets heavily into what you're defining as "usable" and is also more useful to define in terms of cost-per-employee vs total. Obviously a bigger business will have a higher line number total even if the cost per employee is identical, but simultaneously can be expected to be making more revenue to pay for it.
If we're defining an average of a dedicated 5090 pulling 1 kW for every single employee (presumably some people wouldn't use it all the time, but others would then pull the compute for other work), running 24/7 (to cover people running stuff when they're away), then that'd be 8760 kWh per year. At my not particularly cheap New England location that'd be about $1900 per employee per year at the generalized residential rate (~$0.22/kWh), or $156 per month. That doesn't seem radical if it really does boost productivity. However, there is a lot of room to go lower. I'd expect a business to run backup anyway, and these days there are a lot of incentives to do that at least partially with batteries. That also opens up rate shifting as another way to pay back the cost. If we change to time of day pricing, that's 8 hours of peak pricing with the rest off-peak. 8 kWh of battery can now be had for a few thousand. And the off-peak rate is only ~$0.14/kWh, cutting the cost per year by about $700 to $1200 per employee per year. Solar power is also usually far more valuable to use yourself then sell back to the grid, and also continues to plummet in price.
None of this is to say that it makes sense for every place at all, but it's close enough to the the line that the math is at least worth exploring, or could at least lower the cost enough to be worth it given other things. It really comes down to how much extra value the company (or individual) expects to come out of it per month.
>In California or Europe this could be $30-60k per year.
Dunno about Europe, but at the kinda prices I see for California I'm really surprised more places aren't trying to move a lot of usage to battery+renewable.
>The only real moat that local LLMs have right now is privacy.
I don't think resiliency and control are things that can be taken for granted anymore, particularly on the global scale. War and terrorism is getting worse again. International relations are getting nastier, and governments have the power to just order places cut off. If LLMs aren't particularly valuable to a business, then why an expensive subscription? But if they are particularly valuable, then insurance is something leadership should be contemplating.
Earlier quoted context omitted.
Anthropic, OpenAI, GDM, and Meta spend more on training than other labs by an order of magnitude. If they felt safe reducing this spend they would. These labs fear getting outcompeted.
Again, I am talking about inference, not training. Please read.
Earlier quoted context omitted.
Or ready to get a divorce... The load-bearing seam of the relationship is affection versus annoyance.
I think this is happening right now. It's beyond ridiculous and many seem to be praising 5.6 Sol and others, even Grok 4.6 seems to be a breath of fresh air
But the "open" models are right there too, and I'll be using them more in the next weeks as I get better chat interfaces.
I swear, Qwen 3.8 27B @ Q8 is smarter than Sonnet 5 most of the time. Why wouldn’t corporate America self host at this point, especially with better options like Deepseek Flash and GLM 5.3 flash that’s a middle ground between Sonnet and Opus
is this actually the case? I haven't kept up with the small models but if there's roughly Sonnet 4.6 level capable open small models, then I'd be impressed
Earlier quoted context omitted.
I'm not sure why someone hasn't developed a company offering services that distributes AI across all idle or under-utilized VM's and PC's for enterprises in order to serve open sourced models. Outside of the electricity bill, there's no additional expenditure and you get the AI. We've all seen the office spaces where there's 200 empty computers on a floor. Combined, it's something like 500 cores at ~3 Ghz each and ar…
Vibe code that app and business. It’s a great idea like a kind of SETI@Home for business but I’m not sure the compute and latency will really be good enough unless you’re talking about a business with literally thousands of machines sitting half idle and always on, or always mostly on. i imagine it would take maybe 50-100 MacBook Pros running such a service to get to Claude level performance for one or two people.