OpenAI's plans according to Sam Altman

> Whether scaling laws hold or not, is up for debate.

The scaling laws have broken: https://arxiv.org/abs/2210.14891

That doesn't change the fact that these corporations have an interest in data privacy and cost reduction. To wit, if MS wouldn't guarantee data security in european datacenters, they would be way less competitive in the EU.

Many corporations run their own datacenters, have on premises mailservers and run their own backup solutions. Not everything is in the cloud.

> Guess who controls these markets

These markets benefit from deep OS integration, interconnectivity between users products and a long history of "we have always used Word". None of that is the case for LLMs.

> For LLMs, either you supply a cloud API, or deal with a significant deployment burden nobody AFAIK has even started handling.

Why would there be a deployment burden? The actual setup isn't hard, the rest is wrapping the thingamabob into an API the product can use...same btw. as I have to do for cloud-LLM APIs, because the product ideally wants to be able to use different providers, so I have to abstract the API away anyway.

Sure, I have to worry about infrastructure, maintenance, and performance characteristics, but that's hardly different from running our databases or web-backends.

> Current best hope for on-premise is that one of the OS makers decides to do the work and does it well enough we could trust it.

Why? I don't have to trust the OS providers to integrate, say, postgres or nginx or redis either. Why would I need their help to run another piece of software that lives entirely in userspace?

>These markets benefit from deep OS integration, interconnectivity between users products and a long history of "we have always used Word". None of that is the case for LLMs.

The old moats did not stop existing because LLMs came along. e.g. a big use case for LLMs is running over one's own documents. We can guess who very likely already has access (MS and Google), and who is going to get a scary permission prompt at best (everyone else). OS integration is obviously coming, MS announced Copilot and Google can just silently upgrade Assistant. We may not be far from a repeat of the IE saga, where MS declares Copilot as essential part of Windows.

My point is that commercial demand for privacy is not enough, and may not prevent Cloud APIs from dominating.

>>For LLMs, either you supply a cloud API, or deal with a significant deployment burden >Why would there be a deployment burden? The actual setup isn't hard, the rest is wrapping the thingamabob into an API the product can use...

Well, the entire idea is to run locally, right? Now you have to worry about getting the runtime to work on 3 different OSs, and many many hardware variations (like the many Intel AVX variants, low memory configurations, etc.). Just take a look at the recent moves to flatpack/snap everything, and that's merely due to software variations on a single OS.

>Why? I don't have to trust the OS providers to integrate, say, postgres or nginx or redis either. Why would I need their help to run another piece of software that lives entirely in userspace?

I expect LLMs to be a ubiquitous UI paradigm. We do expect OS providers to bundle a UI kit, and not everyone to roll their own, right?

Besides, you are comparing to server software. But non-cloud LLMs are expected to run locally on clients. That's a different world. Clients are a lot more diverse, weaker in processing power, and most can't handle upgrading postgres.

All that said, perhaps we should look at legislation as well as the possibility of a OS provider playing nice. We're talking about a lot of data/power in the hands of a few corps.

>I think we don't need to argue the fact that an on premise solution is cheaper than a cloud solution for a lot of tasks

In the absolute sense where we look at the total cost of running the model (and not care how it's distributed or include profits), you may be right even with scale efficiencies - making a determination requires data about cloud server costs we do not have. But I can make an informed guess about the dollar cost the user sees.

The cost the user sees is influenced by the factor called 'Microsoft and Google (etc.) have a lot of money, and seem to be perfectly willing to absorb costs to control the market and get user data', and that's enough to get user costs very low when calling to Cloud LLMs.

>>>I can easily see scenarious where big tech fail to pivot into AI properly and go down.

>>I don't think you have much to base scenarios where big tech goes down upon, but I'll be glad to hear an argument. Hopefully being based on more than predictions of local LLM performance.

>What else should they be based on, pray, given the fact that smaller models suitable for on-premise and even on-machine use are improving rapidly?

My consistent point is that technical performance is not enough. While the small open models have a tendency to overclaim[0], they'll get to GPT-4 level in time.

Success, however, is not determined by technical performance alone. There are some very big hurdles ahead. Why should big tech fall when they pivoted ahead in time, and maintain some very useful moats?

[0] https://arxiv.org/abs/2305.15717