Earlier quoted context omitted.
The question will be whether customers can switch. Can you install a near-SOTA model on a cluster in a data center? Of course. Compliance and operations are the sticking points. I work in healthcare IT, and it's amazing how tight the data compliance requirements are. I can't have someone in Canada look at prod data. If we told hospitals that we were handing off PHI/PII to Chinese models, they'd end our relationship d…
My primary role is cybersecurity in a regulated entity in a regulated industry, I am highly confident it is straightforward to do so based on work accomplished in only a couple of weeks. Stand up a router, stand up a Kubernetes cluster if you don't have one, stand up the necessary VMs and compute for serving inference. Two pizza team, in my experience. Customers can switch (although we can argue the speed and pain of…
Meanwhile the cost/benefit analysis doesn't move much even if you are paying 2x for tokens, and you don't need anything on prem.