Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is too cheap to beat

generatingconversation.substack.com

Post reply on HN
Print view View original

111–120 of 426 posts

Re: OpenAI is too cheap to beat

#111

Post by sharemywin »

freedomben wrote: post #68
SkyMarshal wrote: post #57

Earlier quoted context omitted.

I'm not saying they will on the high-end, but maybe on the low end. Apple's strategy is to embed local AI in all their devices. Local AI will never be as capable as AI running in massive GPU datacenters, but if it can get to a point that it's "good enough" for most average users, that may be enough for Apple to undercut the low end of the market.

> Local AI will never be as capable as AI running in massive GPU datacenters I'm not sure this is true, even in the short term. For some things yes, that's definitely true. But for other things that are real-time or near real-time where network latency would be unacceptable, we're already there. For example, Google's Pixel 8 launch includes real-time audio processing/enhancing which is made possible by their new Tens…

I wonder if you could send the embeddings or some higher level compressed latent vector across the cloud you couldn't get the best of both worlds.

GPS, phone orientation, last 5 apps you were in, etc. --> embedding

you might even have like "what time is it?" compressed as it's own embedding.

Re: OpenAI is too cheap to beat

#113

Post by dekhn »

shrubble wrote: post #38
dekhn wrote: post #23

Earlier quoted context omitted.

How much does an A100 consume in power a year (in dollar costs)? How much does it cost to hire and retain datacenter techs? How long does it take to expand your fleet after a user says "we're gonna need more A100s?" How many discounts can you get as a premier customer? Answer these questions, and the equation shifts a bunch!

Not really. A full rack with 16 amps usable power and some bandwidth is $400/month in Kansas City, MO. That is enough to power 5x A100s 24x7, so 10k plus $80 per month each, amortized, of course many more A100s would drop the price. Once installed in the rack ($250 1 time cost) you shouldn't need to touch it. So 10k plus $1250 per A100, per year including power. You can put 2 or 3 A100s per cheapo Celeron based CPU w…

Did you also include the network required to make the A100s talk to each other? Both the datacenter network (so the CPUs can load data) and the fabric (so the A100s can talk?)

You also left out the data tech costs- probably at least $50K/individual-year in KC (although I guess I'd just work for free ribs).

If you're putting A100s into celeron motherboards... I don't know what to say. You're not saving money by putting a ferrari engine in a prius.

Re: OpenAI is too cheap to beat

#114

Post by CamperBob2 »

freedomben wrote: post #82
CamperBob2 wrote: post #56

Earlier quoted context omitted.

Just yesterday, while driving: "Read last message." Siri: "Sorry. Dictation service is unavailable at the moment." It's past time for excuses. High-level people at Apple need to be fired over this. Hello? Tim? Do your job. Hello? Anybody home...?

Nobody is switching away from Apple over this, so ultimately Tim is doing his job. Under his watch Apple has become the defacto choice for entire generations. Between vendor-lockin/walled gardens and societal/cultural pressures (don't want to be a green bubble!), they have one of the stickiest user bases there are.

Stop excusing shitty work from trillion-dollar companies. It makes the world a worse place.

Re: OpenAI is too cheap to beat

#115

Post by goosinmouse »

serjester wrote: post #55

I think this is under appreciated. I run a "talk-to-your-files" website with 5ish K MRR and a pretty generous free tier. My OpenAI costs have not exceeded $200 / mo. People talk about using smaller, cheaper models but unless you have strong data security requirements you're burdening yourself with serious maintenance work and using objectively worse models to save pennies. This doesn't even consider OpenAI continuous…

Are you using 3.5 turbo? Its always funny when i test a new fun chatbot or something and see my API usage 10x just from a single GPT 4 API call. Although i only usually have a $2 bill every month from openAI.

Re: OpenAI is too cheap to beat

#116

Post by jaredklewis »

Bukhmanizer wrote: post #103
serjester wrote: post #55

I think this is under appreciated. I run a "talk-to-your-files" website with 5ish K MRR and a pretty generous free tier. My OpenAI costs have not exceeded $200 / mo. People talk about using smaller, cheaper models but unless you have strong data security requirements you're burdening yourself with serious maintenance work and using objectively worse models to save pennies. This doesn't even consider OpenAI continuous…

The bleeding obvious is that OpenAI is doing what most tech companies for the last 20 years have done. Offer the product for dirt cheap to kill off competition, then extract as much value from your users as possible by either mining data or hiking the price. I don’t understand how people are surprised by this anymore. So yeah, it’s the best option right now, when the company is burning through cash, but they’re plann…

> Offer the product for dirt cheap to kill off competition, then extract as much value from your users as possible by either mining data or hiking the price.

Genuine question, what are some examples of companies in that "hiking the price" camp?

I can think of tons of tech companies that sold or sell stuff at a loss for growth, but struggling to find examples where the companies then are able to turn dominant market share into higher prices.

To be clear, I'm definitely not implying they are not out there, just looking for examples.

Re: OpenAI is too cheap to beat

#117

Post by sharemywin »

sidnb13 wrote: post #10
ilaksh wrote: post #4

I think the weird thing about this is that it's completely true right now but in X months it may be totally outdated advice. For example, efforts like OpenMOE https://github.com/XueFuzhao/OpenMoE or similar will probably eventually lead to very competitive performance and cost-effectiveness for open source models. At least in terms of competing with GPT-3.5 for many applications. Also see https://laion.ai/ I also bel…

> I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedback. I guess if we ignore pretraining, don't sample-efficient fine-tuning on carefully curated instruction datasets sort of achieve this? LIMA and OpenOrca show some really promising results to date.

distilbert was trained from Bert. there might be an angle using another model to train the model especially if your trying to get something to run locally.

Re: OpenAI is too cheap to beat

#118

Post by shaburn »

appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

Your Uber/AirBnB/Wework all have physical base units with ascending costs due to inflation and theoretical economies of scale.

AI models have some GPU constraints but could easily reach a state where the cost to opperate falls and becomes relatively trivial with almost no lowerbound, for most use cases.

You are correct there is a race for marketshare. The crux in this case will be keeping it. Easy come, easy go. Models often make the worst business model.

Re: OpenAI is too cheap to beat

#119

Post by blackoil »

appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

This point is discussed in the article. Title is not for Google/Meta, they'll invest all the billions that they have to.

It is for the consumers of these models, is there even a point to train your own or experiment with OSS!

Re: OpenAI is too cheap to beat

#120

Post by loganfrederick »

jaredklewis wrote: post #116
Bukhmanizer wrote: post #103

Earlier quoted context omitted.

The bleeding obvious is that OpenAI is doing what most tech companies for the last 20 years have done. Offer the product for dirt cheap to kill off competition, then extract as much value from your users as possible by either mining data or hiking the price. I don’t understand how people are surprised by this anymore. So yeah, it’s the best option right now, when the company is burning through cash, but they’re plann…

> Offer the product for dirt cheap to kill off competition, then extract as much value from your users as possible by either mining data or hiking the price. Genuine question, what are some examples of companies in that "hiking the price" camp? I can think of tons of tech companies that sold or sell stuff at a loss for growth, but struggling to find examples where the companies then are able to turn dominant market s…

Uber, Netflix and the online content streaming services. These are probably the most prominent examples from this recent 2010s era.
Post reply on HN