Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is too cheap to beat

generatingconversation.substack.com

Post reply on HN
Print view View original

271–280 of 426 posts

Re: OpenAI is too cheap to beat

#271

Post by Havoc »

vlod wrote: post #196
Havoc wrote: post #45

Yep. Building a project that needs some LLMs. I'm very much of the self-hosting mindset so will try DIY, but it's very obviously the wrong choice by any reasonable metric. OpenAI will murder my solution by quality, by availability, by reliability and by scalability...all for the price of a coffee. It's a personal project though & partly intended for learning purposes so there is scope for accepting trainwreck level t…

Isn't it very expensive to do this? Don't you need a jacked up PC with multiple expensive NVidia cards?

A jacked up PC can do really well and there is much fun to be had there.

...but you'd struggle to get close to even GPT 3.5 let alone 4 for generic tasks.

For custom tunes...yeah sure custom rolls will beat generic openAI. But that's a bit like pitting customed tuned cars against street legal manufacturer cars. It's an apple to oranges comparison

Re: OpenAI is too cheap to beat

#272

Post by gms »

somsak2 wrote: post #254
appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

I use Uber in SF all the time, and while it's absolutely more expensive than it was during those go-go years, it's actually an even better experience than it was before (specifically on how fast it is to get a driver near you).

You must be experiencing another SF than me...

Re: OpenAI is too cheap to beat

#273

Post by throwaway50182 »

altdataseller wrote: post #237
throwaway50182 wrote: post #233

Earlier quoted context omitted.

Well I'd have to change my Terraform provider and the managed Kubernetes resource... Other than that, it'd be the same. So half an hour of coding + half an hour of reconfiguring CI secrets?

You lost me at Terraform and Kubernetes..

What's a serious cloud provider-portable replacement for Terraform?

And Kubernetes? It goes way beyond containers, you can replace many cloud provider-specific resources with Kubernetes resources. What alternative gives you that?

Re: OpenAI is too cheap to beat

#274

Post by borg16 »

eek2121 wrote: post #250
user3939382 wrote: post #223

Earlier quoted context omitted.

> The winner is going to be the consumers Cloud infra may be a comparable market, since computation is a big share of AI costs. Did consumers win big from competition between AWS, Azure, and GCP? Not sure. I see an uptick in write ups saying “We switched off cloud and reduced costs by 2/3rds.” Not a scientific sample but may leave the question open.

Can confirm: The cloud computing fad is well underway to dying...that is why AI is booming. One need only to follow the wall street dollars to figure that one out. Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Cloud computing never was good in terms of value, however, it was only good in terms of scalability. AI solutions built on top of 'the cl…

> Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions.

Could you provide some examples?

Re: OpenAI is too cheap to beat

#275

Post by throwaway50182 »

jokethrowaway wrote: post #256
vikramkr wrote: post #241

Earlier quoted context omitted.

There are a HUGE number of startups that would have had a massively harder time if they had to roll their own infra. Who cares if the companies eventually have to move off? (Though even Netflix seems ok with it overall for now). There are a ton of services that just wouldn't exist otherwise

I disagree, at early startup scale you don't need much, you just buy a better VPS when you need to scale up Learning aws vs learning how to operate a vps are of comparable complexity It took me less then a day to setup infra for my startup (more than 10 years ago)

If you're just running VPS on the big clouds, you're not going to get much advantage out of it, indeed.

But tell me what third tier cheap provider has managed scale-to-zero-or-infinity functions? Managed storage with S3-like API? Where can I get an API gateway cheaper than Amazon? What about managed databases? These tools allow me to develop insanely scalable software incredibly easily.

Agreed - all the big clouds are very expensive VPS hostings. Don't use it for that.

Re: OpenAI is too cheap to beat

#276

Post by pleoxy »

vikramkr wrote: post #241
user3939382 wrote: post #223

Earlier quoted context omitted.

> The winner is going to be the consumers Cloud infra may be a comparable market, since computation is a big share of AI costs. Did consumers win big from competition between AWS, Azure, and GCP? Not sure. I see an uptick in write ups saying “We switched off cloud and reduced costs by 2/3rds.” Not a scientific sample but may leave the question open.

There are a HUGE number of startups that would have had a massively harder time if they had to roll their own infra. Who cares if the companies eventually have to move off? (Though even Netflix seems ok with it overall for now). There are a ton of services that just wouldn't exist otherwise

It's not that hard. Particularly for simple deployments, which a startup should have or they are doing it wrong.

Re: OpenAI is too cheap to beat

#277

Post by vikramkr »

pleoxy wrote: post #276
vikramkr wrote: post #241

Earlier quoted context omitted.

There are a HUGE number of startups that would have had a massively harder time if they had to roll their own infra. Who cares if the companies eventually have to move off? (Though even Netflix seems ok with it overall for now). There are a ton of services that just wouldn't exist otherwise

It's not that hard. Particularly for simple deployments, which a startup should have or they are doing it wrong.

How would you do it for a simple webapp? Genuine question

Re: OpenAI is too cheap to beat

#278

Post by garciasn »

borg16 wrote: post #274
eek2121 wrote: post #250

Earlier quoted context omitted.

Can confirm: The cloud computing fad is well underway to dying...that is why AI is booming. One need only to follow the wall street dollars to figure that one out. Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Cloud computing never was good in terms of value, however, it was only good in terms of scalability. AI solutions built on top of 'the cl…

> Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Could you provide some examples?

Dropbox has been the biggest name who did the whole “cloud repatriation” thing, which was all the rage at the beginning of 2023, with claims that the cloud was soon to be dead—but cloud revenues are supposedly expected to be in excess of $1T by 2026, so whatever.

Some random survey from ESG found 60% of respondents repatriated at least some workloads. Who knows what the N was though.

Source: https://www.sdxcentral.com/articles/thinking-about-cloud-rep...

Frankly, it makes sense that companies are deciding what works best and where. But the death knell of cloud providers is simply not happening.

Re: OpenAI is too cheap to beat

#279

Post by [deleted] »

advaith08 wrote: post #60
sidnb13 wrote: post #6

Earlier quoted context omitted.

Yep, batching is a feature I really wish the OpenAI API had. That and the ability to intelligently cache frequently used prompts. Much easier to achieve this with a hosted OS model, so I guess it's a speed + customizability/cost tradeoff for the time being.

imo they dont have batching because they pack sequences before passing through the model. so a single sequence in a batch on OpenAI might have requests from multiple customers in it

[deleted]

Re: OpenAI is too cheap to beat

#280

Post by vikramkr »

jokethrowaway wrote: post #256
vikramkr wrote: post #241

Earlier quoted context omitted.

There are a HUGE number of startups that would have had a massively harder time if they had to roll their own infra. Who cares if the companies eventually have to move off? (Though even Netflix seems ok with it overall for now). There are a ton of services that just wouldn't exist otherwise

I disagree, at early startup scale you don't need much, you just buy a better VPS when you need to scale up Learning aws vs learning how to operate a vps are of comparable complexity It took me less then a day to setup infra for my startup (more than 10 years ago)

Sorry not sure if I'm missing something obvious but isn't a VPS gonna be hosted by a cloud provider? How would that be an alternative to using the cloud?
Post reply on HN