Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is too cheap to beat

generatingconversation.substack.com

Post reply on HN
Print view View original

391–400 of 426 posts

Re: OpenAI is too cheap to beat

#391

Post by flir »

visarga wrote: post #344
flir wrote: post #294

Earlier quoted context omitted.

The data is the moat. (If you can train your internally deployed LLM on data none of your competitors have, that's an advantage).

It's not anymore. If the model is publicly accessible, its skills can be distilled by performing some API calls and recording input-output pairs. This scheme works so well it has become the main mode to prepare data for small models. Model skills leak.

I agree, publicly deployed models seem to be easy to train from. I did say "internally deployed LLM" though. agentcoops said "...where the models in question increase the productivity of workers in their non-ai-related profit centers" above, that's the bit I was thinking about. I think private models, either trained from scratch or fine-tuned, are going to be a big deal though they won't make the PR splash that public models make.

Re: OpenAI is too cheap to beat

#392

Post by intended »

matusp wrote: post #368
intended wrote: post #340

Earlier quoted context omitted.

If you swap an LLM, at the very least, you have to run your entire Eval set again. This will almost certainly lead to a prompt rebuild, to better accommodate new model idiosyncrasies. If you are unlucky- your use case may be one where Evals require human review. Unless you are YOLOing it without evals. In that case this is relevant.

You need to rerun eval after each LLM update. GPTs have a new version every couple of months and their capabilities can change quite drastically pretty much randomly. Maybe they will make it more robust in time, but I think this is the feature of the technology and people will have to adapt to these quirks

Yup. Constant Eval.

Re: OpenAI is too cheap to beat

#393

Post by chpatrick »

TeMPOraL wrote: post #390
chpatrick wrote: post #375

Earlier quoted context omitted.

Not exactly but it would be a good name for it! If you run it you get some money and higher balances mean better treatment when it takes over.

Assuming the baseline treatment is very bad, this is exactly what Roko's basilisk is.

RoCoin

Distributed evil AI evaluation in exchange for protection from it. You can trade the protection on the free market.

Re: OpenAI is too cheap to beat

#394

Post by nvm0n2 »

visarga wrote: post #344
flir wrote: post #294

Earlier quoted context omitted.

The data is the moat. (If you can train your internally deployed LLM on data none of your competitors have, that's an advantage).

It's not anymore. If the model is publicly accessible, its skills can be distilled by performing some API calls and recording input-output pairs. This scheme works so well it has become the main mode to prepare data for small models. Model skills leak.

The conclusion for that seems to be that it just yields a model that has the surface look and feel of GPT3 or 4 but without the depth, so the experience quickly becomes unsatisfactory once you go out of the fine tuning dataset.

Re: OpenAI is too cheap to beat

#395

Post by nvm0n2 »

SupremumLimit wrote: post #295
eek2121 wrote: post #250

Earlier quoted context omitted.

Can confirm: The cloud computing fad is well underway to dying...that is why AI is booming. One need only to follow the wall street dollars to figure that one out. Several very big profile names have recently begun moving back to self-hosted, hybrid, or dedicated hosting solutions. Cloud computing never was good in terms of value, however, it was only good in terms of scalability. AI solutions built on top of 'the cl…

Which cloud provider is it, if you don’t mind sharing?

Hetzner or Oracle, maybe?

Re: OpenAI is too cheap to beat

#397

Post by shmel »

appplication wrote: post #106

The premise of this is flawed. OpenAI is cheap because of has to be right now. They need to establish market dominance quickly, before competitors slide in. The winner of this horse race is not going to be the company with the best performing AI, it’s going to be the one who does the best job at creating an outstanding UX, ubiquitously presence, entrenching users, and building competitive moats that are not feature d…

I think the main difference is that OpenAI doesn't have network effect. AirBnB/Uber/any social network wins because you want to be where everyone else is. ChatGPT is great, but I can switch tomorrow to something else without any issues.

Re: OpenAI is too cheap to beat

#398

Post by pleoxy »

devbent wrote: post #364
pleoxy wrote: post #336

Earlier quoted context omitted.

Ingress that handles SSL. nginx, or caddy. Then stand up your app server behind that on the same VM. Database can be on the same or different VM. I try to not use anything else if I can avoid it on a new project. Ingress gives you the ability to load balance and is threaded and will scale with network transfer and cpu. Database should scale with a bump in VM specs as well, CPU and disk IOPS. If you keep your app serv…

> If you keep your app server stateless you can simply multiply it for the number of copies you need for your load. This right here so many times over. I'm not going to spend time worrying about scaling, I'm going spend time figuring out how to make stuff stateless.

Shove your state into the database. Done.

Re: OpenAI is too cheap to beat

#399

Post by pleoxy »

throwaway50182 wrote: post #362
pleoxy wrote: post #281

Earlier quoted context omitted.

You get portability. Which the functions do not provide. Open source solutions have a longer career utility than proprietary offerings. I remember when NetWare certs were all the rage. Useless now. I remember msce. But if you learned open tools 35 years ago instead... You get the picture. You can scale from 1 thread 512MB RAM, to 500 threads and 12TB of RAM (off the shelf). Which is good enough for almost everyone wh…

I agree completely, but the argument remains the same - there's not much utility in using the big clouds as VPS providers, and it's definitely costly to do so.

That part of true. Using EC2 at AWS now and it's much more expensive than vultr, Hetzner, or akamai cloud.

Re: OpenAI is too cheap to beat

#400

Post by htrp »

edmundsauto wrote: post #319
shon wrote: post #311

Earlier quoted context omitted.

In that case, X.AI, powered by X/Twitter/Tesla data and possibly Facebook (both closed, and somewhat hard to crawl inside) have the largest moat.

I don’t think they necessarily will be allowed to train on their data unless they get explicit permission. They will try, but the way I see privacy revaluations is that users will have to authorize specific uses of their data and not be surprised by any application. This could be one of the more interesting privacy fights of the next decade. I’m sure there are easy cynical takes about how they will just shrink wrap t…

> I don’t think they necessarily will be allowed to train on their data unless they get explicit permission. They will try, but the way I see privacy revaluations is that users will have to authorize specific uses of their data and not be surprised by any application.

If there's one company that I don't think cares about user permissions or the law, it'd be Twitter.

The EU officially warned Elon about DSA fines and the response was less than serious.

https://www.cnn.com/2023/10/10/tech/x-europe-israel-misinfor...

Post reply on HN