Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is too cheap to beat

generatingconversation.substack.com

Post reply on HN
Print view View original

361–370 of 426 posts

Re: OpenAI is too cheap to beat

#361

Post by Cwizard »

wrsh07 wrote: post #301
KaiserPro wrote: post #207

Earlier quoted context omitted.

the policy says that, but there is no external verification or auditing. So its hard to trust.

They have a soc 2 so literally an (external) auditor has looked at their data retention policies and if your business is a customer you should request access to the report https://trust.openai.com/

but does that matter legally for health, finance and legal sectors? I am not familiar with the laws themselves but I worked in finance for a long time and the internal rules where that sensitive data cannot move off premises no matter what the external party promised/had certified.

Re: OpenAI is too cheap to beat

#362

Post by throwaway50182 »

pleoxy wrote: post #281
throwaway50182 wrote: post #275

Earlier quoted context omitted.

If you're just running VPS on the big clouds, you're not going to get much advantage out of it, indeed. But tell me what third tier cheap provider has managed scale-to-zero-or-infinity functions? Managed storage with S3-like API? Where can I get an API gateway cheaper than Amazon? What about managed databases? These tools allow me to develop insanely scalable software incredibly easily. Agreed - all the big clouds ar…

You get portability. Which the functions do not provide. Open source solutions have a longer career utility than proprietary offerings. I remember when NetWare certs were all the rage. Useless now. I remember msce. But if you learned open tools 35 years ago instead... You get the picture. You can scale from 1 thread 512MB RAM, to 500 threads and 12TB of RAM (off the shelf). Which is good enough for almost everyone wh…

I agree completely, but the argument remains the same - there's not much utility in using the big clouds as VPS providers, and it's definitely costly to do so.

Re: OpenAI is too cheap to beat

#363

Post by PunchTornado »

outlace wrote: post #185
appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

But OpenAI appears to have some sort of data moat. I doubt their model is the best in the world, but more/better data generally beats better model, and GPT-4 definitely beats Claude, Bard, Bernie and the rest probably because they curated the best quality and largest data set. Maybe that moat doesn't last long but perhaps they have exclusive rights to some of that dataset through commercial agreements that could be a…

I was using Bard and chatGPT in parallel, but lately I just default to Bard. To me it's a better model with much more accurate answers, while chatGPT just gives you bombastic words.

Re: OpenAI is too cheap to beat

#364

Post by devbent »

pleoxy wrote: post #336
vikramkr wrote: post #277

Earlier quoted context omitted.

How would you do it for a simple webapp? Genuine question

Ingress that handles SSL. nginx, or caddy. Then stand up your app server behind that on the same VM. Database can be on the same or different VM. I try to not use anything else if I can avoid it on a new project. Ingress gives you the ability to load balance and is threaded and will scale with network transfer and cpu. Database should scale with a bump in VM specs as well, CPU and disk IOPS. If you keep your app serv…

> If you keep your app server stateless you can simply multiply it for the number of copies you need for your load.

This right here so many times over.

I'm not going to spend time worrying about scaling, I'm going spend time figuring out how to make stuff stateless.

Re: OpenAI is too cheap to beat

#365

Post by olivermuty »

eek2121 wrote: post #250
user3939382 wrote: post #223

Earlier quoted context omitted.

> The winner is going to be the consumers Cloud infra may be a comparable market, since computation is a big share of AI costs. Did consumers win big from competition between AWS, Azure, and GCP? Not sure. I see an uptick in write ups saying “We switched off cloud and reduced costs by 2/3rds.” Not a scientific sample but may leave the question open.

Can confirm: The cloud computing fad is well underway to dying...that is why AI is booming. One need only to follow the wall street dollars to figure that one out. Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Cloud computing never was good in terms of value, however, it was only good in terms of scalability. AI solutions built on top of 'the cl…

Just gonna chip in here and say that anything costing a 100 bucks is not valid as an argument in a conversation about cloud.

The prime selling point for cloud for large enterprises was (and still is):

- a signatory that shares blame on several core security issues (iso stuff) - high amount of flexibility for individual teams used to asking for a vm then waiting four weeks for the itops dept to bring it online

Now the vast majority of cloud moves for large enterprises ends up as a shitshow due to poor implementation sure, but the key points for getting it sold are still there.

And ofc you CAN still cost optimize with cloud, its just harder.

Context: Worked in post and pre sales over some years in MSFT in the enterprise segment.

I got out before the downturn and everyone talking about cost, but my approach in selling azure to the c-suite would be fairly similar today I reckon.

Re: OpenAI is too cheap to beat

#366

Post by Cholical »

charlesischuck wrote: post #109
joefourier wrote: post #48

Earlier quoted context omitted.

AWS is extremely overpriced for nearly every service. I don’t know why anyone else outside of startups with VC money to burn or bigcos that need the “no one ever got fired for buying IBM” guarantee would use them. You’re better off with Lambdalabs or others which charge only $1.1/h per A100. Also that is a 8xA100 system as others have noted, but it is the 40GB one which can be found on eBay for as low as $3k if you g…

You pay for the system not the gpu with AWS. It's absolutely worth the money when you look at the whole picture. Also lambda labs never has availability. I actually can schedule a distributed cluster on AWS.

We built a tool for lambda labs and other clouds that launches a specific instance whenever it becomes available and notifies you. We poll Lambda Labs for availability every 3 seconds. Would this be something that would be useful for you?

Re: OpenAI is too cheap to beat

#367

Post by chpatrick »

TotalCrackpot wrote: post #348
NikolaNovak wrote: post #339

Earlier quoted context omitted.

Possibly unrealistic, but my fear is that they will end up like Netflix more than uber. Some will scream in horror but I wanted Netflix to be a monopoly. A single place and app and account with all the content I need. "competition" in streaming space has been nothing but disastrous for me as a consumer. It led to greedy heterogeneous islands of content, with proliferation of crappy apps and pointless restrictions and…

Just like the pirate Bay online cinema is better than every streaming service there could be pirate LLM that uses all the data, maybe it could be even trained by internet users sharing a bit of compute with some program?

Or a cryptocurrency where the proof of work is running a malicious AI...

Re: OpenAI is too cheap to beat

#368

Post by matusp »

intended wrote: post #340
adam_arthur wrote: post #231

Earlier quoted context omitted.

Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB? Margin is a function of stickiness/cost of switching (among other things). I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off…

If you swap an LLM, at the very least, you have to run your entire Eval set again. This will almost certainly lead to a prompt rebuild, to better accommodate new model idiosyncrasies. If you are unlucky- your use case may be one where Evals require human review. Unless you are YOLOing it without evals. In that case this is relevant.

You need to rerun eval after each LLM update. GPTs have a new version every couple of months and their capabilities can change quite drastically pretty much randomly. Maybe they will make it more robust in time, but I think this is the feature of the technology and people will have to adapt to these quirks

Re: OpenAI is too cheap to beat

#369

Post by golergka »

chpatrick wrote: post #367
TotalCrackpot wrote: post #348

Earlier quoted context omitted.

Just like the pirate Bay online cinema is better than every streaming service there could be pirate LLM that uses all the data, maybe it could be even trained by internet users sharing a bit of compute with some program?

Or a cryptocurrency where the proof of work is running a malicious AI...

You mean the basilisk?

Re: OpenAI is too cheap to beat

#370

Post by golergka »

beezlebroxxxxxx wrote: post #123
jaredklewis wrote: post #116

Earlier quoted context omitted.

> Offer the product for dirt cheap to kill off competition, then extract as much value from your users as possible by either mining data or hiking the price. Genuine question, what are some examples of companies in that "hiking the price" camp? I can think of tons of tech companies that sold or sell stuff at a loss for growth, but struggling to find examples where the companies then are able to turn dominant market s…

Uber is probably the biggest pure example. When I was in uni when they first spread, Uber's entire business model was flood the market with hilariously low prices and steep discounts. People overnight started using them like crazy. They were practically giving away their product. Now, they're as expensive, if not sometimes more expensive, than any other taxi or ridesharing service in my area. One thing I'll add is th…

"Uber/Airbnb is expensive now" is an entirely american phenomena. In Europe and Latin America, both are still cheaper than alternative (comparable hotels and yellow cabs). Most likely in other parts of the world too.
Post reply on HN