OpenAI is too cheap to beat • Hacker News Board

Re: OpenAI is too cheap to beat

#21

Post by vsreekanti » Thu, Oct 12, 2023, 6:53 PM UTC

SkyMarshal wrote: post #15

I think OpenAI may eventually have to go upmarket, as basic "good enough" AI becomes increasingly viable and cheap/free on consumer level devices, supplied by FOSS models and apps. Apple may be leading the way here, with Apple Silicon prioritizing AI processing and built into all their devices. These capabilities are free (or at least don't require an extra sub), and just used to sell more hardware. OpenAI is clearly…

Yep, we agree that the obvious direction of innovation for OSS models is smaller and cheaper, likely at roughly the same quality: https://generatingconversation.substack.com/p/open-source-ll...

Re: OpenAI is too cheap to beat

#22

Post by vsreekanti » Thu, Oct 12, 2023, 6:54 PM UTC

jonplackett wrote: post #3

Is this a reflection of OpenAI’s massive scale making it so cheap for them? Or is it the deal with Microsoft for cloud services making it cheap? Or are they just operating at a massive loss to kill off other competition? Or something else?

Probably some combination of all the above! I think 1 and 2 are interlinked though — the cheaper they can be, the more they build that moat. They might be eating the cost on these APIs too, but unlike the Uber/Lyft war, it'll be way stickier.

Re: OpenAI is too cheap to beat

#23

Post by dekhn » Thu, Oct 12, 2023, 6:55 PM UTC

daft_pink wrote: post #7

I’m confused don’t a100s cost 10,000 to buy? Why would you pay 166k per year to rent?

How much does an A100 consume in power a year (in dollar costs)? How much does it cost to hire and retain datacenter techs? How long does it take to expand your fleet after a user says "we're gonna need more A100s?" How many discounts can you get as a premier customer?

Answer these questions, and the equation shifts a bunch!

Re: OpenAI is too cheap to beat

#24

Post by [deleted] » Thu, Oct 12, 2023, 6:56 PM UTC

sidnb13 wrote: post #17

daft_pink wrote: post #7

I’m confused don’t a100s cost 10,000 to buy? Why would you pay 166k per year to rent?

I would assume the datacenter and infra needed would also contribute a sizeable chunk to the costs when you consider upkeep to run it 24/7

[deleted]

Re: OpenAI is too cheap to beat

#25

Post by sarchertech » Thu, Oct 12, 2023, 6:58 PM UTC

SkyMarshal wrote: post #9

jonplackett wrote: post #3

Is this a reflection of OpenAI’s massive scale making it so cheap for them? Or is it the deal with Microsoft for cloud services making it cheap? Or are they just operating at a massive loss to kill off other competition? Or something else?

Probably the first two, plus first-mover brand recognition. Millions of $20 monthly subs for GPT4 add up. They might also be operating at a loss afaik, but I suspect they're one of the few that can break even just based on scale, brand recognition, and economics.

I haven’t heard any evidence that they have millions of Plus subscribers.

I’ve seen 100 to 200 million active users, but nothing about paid users from them. The surveys I saw when doing a quick google search reported much less than 1% of users paying.

Re: OpenAI is too cheap to beat

#26

Post by nico » Thu, Oct 12, 2023, 6:59 PM UTC

ilaksh wrote: post #4

I think the weird thing about this is that it's completely true right now but in X months it may be totally outdated advice. For example, efforts like OpenMOE https://github.com/XueFuzhao/OpenMoE or similar will probably eventually lead to very competitive performance and cost-effectiveness for open source models. At least in terms of competing with GPT-3.5 for many applications. Also see https://laion.ai/ I also bel…

> I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedback

This makes a lot of sense. A small model that “knows” enough English and a couple of programming languages should be enough for it to replace something like copilot, or use plug-ins or do RAG on a substantially larger dataset

The issue right now is that to get a model that can do those things, the current algorithms still need massive amounts of data, way more than what the final user needs

Re: OpenAI is too cheap to beat

#27

Post by mensetmanusman » Thu, Oct 12, 2023, 7:00 PM UTC

SkyMarshal wrote: post #15

I think OpenAI may eventually have to go upmarket, as basic "good enough" AI becomes increasingly viable and cheap/free on consumer level devices, supplied by FOSS models and apps. Apple may be leading the way here, with Apple Silicon prioritizing AI processing and built into all their devices. These capabilities are free (or at least don't require an extra sub), and just used to sell more hardware. OpenAI is clearly…

Please Apple let me replace worthless Siri with ChatGPT on my iPhone.

Would completely change how I use the device.

Re: OpenAI is too cheap to beat

#28

Post by grahamplace » Thu, Oct 12, 2023, 7:00 PM UTC

SkyMarshal wrote: post #15

I think OpenAI may eventually have to go upmarket, as basic "good enough" AI becomes increasingly viable and cheap/free on consumer level devices, supplied by FOSS models and apps. Apple may be leading the way here, with Apple Silicon prioritizing AI processing and built into all their devices. These capabilities are free (or at least don't require an extra sub), and just used to sell more hardware. OpenAI is clearly…

> OpenAI is clearly going to compete in that market with its upcoming phone

What phone are you referring to? A quick google didn’t seem to pull up anything related to OpenAI launching a hardware product?

Re: OpenAI is too cheap to beat

#29

Post by awesomeMilou » Thu, Oct 12, 2023, 7:02 PM UTC

SkyMarshal wrote: post #15

I think OpenAI may eventually have to go upmarket, as basic "good enough" AI becomes increasingly viable and cheap/free on consumer level devices, supplied by FOSS models and apps. Apple may be leading the way here, with Apple Silicon prioritizing AI processing and built into all their devices. These capabilities are free (or at least don't require an extra sub), and just used to sell more hardware. OpenAI is clearly…

> OpenAI is clearly going to compete in that market with its upcoming phone.

Excuse me, I'm not an english native, you mean like a smart phone? Or do you mean some sort of other new business direction? Where did you get the info thtat they're planning to launch a phone?

Re: OpenAI is too cheap to beat

#30

Post by etothepii » Thu, Oct 12, 2023, 7:02 PM UTC

latchkey wrote: post #20

daft_pink wrote: post #7

I’m confused don’t a100s cost 10,000 to buy? Why would you pay 166k per year to rent?

For the same reason people use AWS. Spending the capex/opex to run a cluster of compute isn't easy or cheap. It isn't just the cost of the GPU, but the cost of everything else around it that isn't just monetary.

This could be an interesting comparison. My experience with AWS is that it was super easy and cheap to start on. By the time we could use whole servers we were using so much AWS orchestration that it's going to be put off until we are at least $1M ARR, and probably til we are at $5M.

Make adoption easy, give a free base tier but charge more could be a very effective model to get start ups stuck on you. It even probably makes adoption by small teams in big companies possible that can then grow ...