Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI subscriptions are a ticking time bomb for enterprise

thestateofbrand.com

Post reply on HN
Print view View original

201–210 of 426 posts

Re: AI subscriptions are a ticking time bomb for enterprise

#201

Post by nine_k »

harrall wrote: post #190
crazygringo wrote: post #108

Earlier quoted context omitted.

> within a few years we will be running local models as good as today’s frontier models with almost no cost burden Based on what? The RAM requirements alone are extraordinary. No, running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future.

You can now buy 128 GB unified memory computers from AMD as commodity. They’re still pricey, the world is still scaling up memory production, and a lot of code isn’t yet built for AMD, but we went from the Wright’s brothers first airplane to jet engines in 27 years. I’m not sure “it’s only a few years away” but we are sure moving there fast.

> first airplane to jet engines in 27 years.

Nitpick: more like 36 years, from Wright Flyer in 1903 to Heinkel 178 in 1939. Still quite impressive.

Re: AI subscriptions are a ticking time bomb for enterprise

#203

Post by tyre »

HarHarVeryFunny wrote: post #107
radicalbyte wrote: post #102

Earlier quoted context omitted.

Something I have noticed is that the people who are using it to write everything are the same people who had a poor level of English writing a year or two ago. It's just "intellectual" botox.

Would make sense ... writing is a skill, and one that I think most people are proud of if they are good at it. Maybe it's different if you are doing technical/commercial writing, but for social media where you are writing for fun, and to express yourself, it'd be odd to let AI be your voice unless you realize your own writing is very poor.

> for social media where you are writing for fun, and to express yourself, it'd be odd to let AI be your voice unless you realize your own writing is very poor.

A lot of people post for clout, so something that can skip the difficult process of becoming a good writer (and original thinker) is more than enough. They can churn out think pieces about any topic at an unlimited pace, basically.

It doesn’t add much to the world, but they get a lot of traction (which I cannot understand, given the quality of content.) And that’s what matters to them.

I think if you gave most people the choice between (a) being a thoughtful and original writer (b) being seen as a thoughtful and original writer, the vast majority choose (b). Especially when it is zero effort.

Re: AI subscriptions are a ticking time bomb for enterprise

#204

Post by salawat »

jazzyjackson wrote: post #169
leptons wrote: post #156

Earlier quoted context omitted.

>running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future. That is only true right now because hundreds of billions of dollars are being burned by these AI companies to try to win market share. If you paid what it actually cost, your comment would likely be very different.

No, it's economies of scale and I don't understand where anyone is coming from that thinks they'll be better off buying their own hardware, why would you get a better deal on MATMULs/watt than the cloud providers ?

Another victim of Goldratt's Theory of Constraints. Some things are more important to optimize for than MATMULs per Watt. What that is I leave as an exercise to the student. May you realize what it is before it is too late.

Re: AI subscriptions are a ticking time bomb for enterprise

#205

Post by jmount »

evo_9 wrote: post #79

Every AI subscription is a ticking time bomb for the frontier provider; within a few years we will be running local models as good as today’s frontier models with almost no cost burden. The floor will fall out of the enterprise market for all the frontier companies.

I think this is a good under-represented point. Again and again things that could only run on a mainframe get ported to the personal device level. However it looks like the campaign to eliminate the PC (by pre-buying all RAM) is the counter-stroke.

Re: AI subscriptions are a ticking time bomb for enterprise

#206

Post by majormajor »

stingraycharles wrote: post #182
majormajor wrote: post #161

Earlier quoted context omitted.

Running local applications is less efficient than thin clients to the cloud generally, not just in LLMs. The trick is that you can get to the point where it's effective enough, and affordable enough, that the control and availability factors become dominant.

My point is that you will always get much more value / $ by using cloud based solutions.

I just don't see how that's different from getting more value by giving all your employees the most stripped-down chromebook-type devices and running everything else in the cloud, than by giving them "proper" laptops with local apps.

It's a measure of a very thin sort of "value/$" that excludes a lot of other things that could be of value to a business, like control, predictability, and availability.

Thin clients have been going away for a long time. The trend has been to continue to push higher levels of compute into ever-smaller and ever-more-portable devices.

Re: AI subscriptions are a ticking time bomb for enterprise

#207

Post by Traubenfuchs »

harrall wrote: post #190
crazygringo wrote: post #108

Earlier quoted context omitted.

> within a few years we will be running local models as good as today’s frontier models with almost no cost burden Based on what? The RAM requirements alone are extraordinary. No, running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future.

You can now buy 128 GB unified memory computers from AMD as commodity. They’re still pricey, the world is still scaling up memory production, and a lot of code isn’t yet built for AMD, but we went from the Wright’s brothers first airplane to jet engines in 27 years. I’m not sure “it’s only a few years away” but we are sure moving there fast.

I believe the same thing but keep repeating the question: Then what are all the datacenters for?

Re: AI subscriptions are a ticking time bomb for enterprise

#208

Post by scheme271 »

leptons wrote: post #156
crazygringo wrote: post #108

Earlier quoted context omitted.

> within a few years we will be running local models as good as today’s frontier models with almost no cost burden Based on what? The RAM requirements alone are extraordinary. No, running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future.

>running large models on shared, dedicated hosted hardware at full utilization is going to be vastly more cost-efficient for the foreseeable future. That is only true right now because hundreds of billions of dollars are being burned by these AI companies to try to win market share. If you paid what it actually cost, your comment would likely be very different.

We don't know the parameters but it probably takes at least a H100 and possibly several to run a SOTA model. Given the pricing (25+k per H100 + hardware to run it) and power (700W per H100 + hardware to run it), I don't see how anyone except for a largish company can afford to run this.

Re: AI subscriptions are a ticking time bomb for enterprise

#209

Post by new_account_100 »

niekkamer wrote: post #191
yalogin wrote: post #175

If it’s replacing developers it makes sense to cost more than 20 or 100 per month. The real issue for these llm companies is that they are yet to show value in other areas. Without that they will be relegated to just coding. That is the rush right now for them. What other workflows can they automate. I guess every paperwork can be automated. Once the other areas are developed they will switch the pricing model

They are kinda trying to replace figma. That could be valuable.

IMO the LLM technology is so poor when it comes to converting text descriptions to visual layout that I can't imagine it could possibly succeed as part of a paid design product.

Re: AI subscriptions are a ticking time bomb for enterprise

#210

Post by chris_money202 »

Traubenfuchs wrote: post #207
harrall wrote: post #190

Earlier quoted context omitted.

You can now buy 128 GB unified memory computers from AMD as commodity. They’re still pricey, the world is still scaling up memory production, and a lot of code isn’t yet built for AMD, but we went from the Wright’s brothers first airplane to jet engines in 27 years. I’m not sure “it’s only a few years away” but we are sure moving there fast.

I believe the same thing but keep repeating the question: Then what are all the datacenters for?

Agents
Post reply on HN