Elevated error rate across multiple models

There is something to be said for how the technology stack keeps growing for businesses and what this might mean for the future.

Thirty years ago, you had an OS and you installed applications. No problem.

Later, you had to build and use apps on the internet, an infrastructure that is susceptible to DDOS attacks, government firewalls, and other security risks. Still fine, sort of.

Now, you not only have to build apps on the internet, you also have use LLMs to build apps to remain competitive with other developers. Future (human) maintainers of your code might not properly understand how it works, and if the providers of the LLMs screw up or go rogue, you are properly fucked.

There is a dependency/technology stack debt that is creating risks that need to be acknowledged.

I have been developing software since the late 80s, mostly CAM software for metal cutting machines, and I have been refereeing tabletop roleplaying games like Dungeons & Dragons since the late 70s.

I get the power of LLMs, and I do find them useful. But I find them useful in much the same way I find a really good set of random tables useful, or a good set of rules for procedurally generating something like a star sector for a science fiction campaign.

For my day job developing software, and for the RPG campaigns and books I run and publish today, LLMs are, in many cases, random tables on steroids. After using them for two years, even with all their improvements, I am continually reminded by the results I get that, at the heart of it, I am still dealing with what amounts to randomly generated content.

Yes, I know it is more accurate to call the process probabilistic rather than random. And yes, somebody can construct a technically deterministic setup with fixed weights, fixed seeds, fixed sampling parameters, and a frozen runtime environment. But that is like saying you can recreate a rainstorm if you get a thousand butterflies to flap their wings in exactly the right way. It may be technically true, but it is not how the technology behaves in normal day-to-day use.

For practical purposes, given the same prompt and the same apparent starting conditions, the result can differ each time you use a model. The outputs will often be highly correlated, and often useful, but they are not deterministic software in the ordinary sense.

So far, I am failing to see how the inherent probabilistic nature of the technology can be fully overcome. I understand how we got to where we are today from older neural net technology, including the systems used for vision and sound. What we have now can be very useful. But my view is that it is being badly oversold and overhyped. Its probabilistic nature is being vastly underestimated, and that is a major reason for much of the weirdness and many of the failures we keep seeing.

In tabletop roleplaying, there have been times when hobbyists relied too much on procedurally generated content and ultimately got burned by it, either through campaigns that were not as fun or products that were subpar. Each time, the lesson was the same: there is no substitute for human judgment.

Any workflow or technology incorporating LLMs has to keep humans in the loop, and not merely as rubber stamps. The human has to remain the primary decision maker.

> Any sensible employer will do the same.

Hard to do when each individual provider wants to lock your company into multiyear enterprise contracts.

Yep, that's not an excuse. Claude goes down all the time, should pi also go down?

Oh wait (from another comment under this article): > https://pi.dev/models is throwing an internal server error for me.

maybe they ask a secondary agentic system to fix it. will that be the future of “redundancy”?