Software 3.1? – AI Functions

> run the AI function once it would be much simpler to cache the resulting code than repeatedly re-generate it?

Surely, you'll run a function that does an AI call to cache the resulting code.

Haven’t we been seeing libraries that implement this pattern going on two years now? Take the docstring and monkey patch the function with llm generated code, with optional caching against an AST hash key.

The reason it hasn’t take off is that it’s a supremely bad and unmaintable idea. It also just doesn’t work very well because the LLM doesn’t have access to the rest of the codebase without an agentic loop to ground it.

Apparently we have blogging-3.0 as well, since the article is littered with AI-isms.

These attempts at generating code that adheres to a whatever spec in Python of all languages are futile and just please investors.

There is a reason that really proving adherence to a spec or making arguments that the spec is reasonable in the first place is hard.

But hey, thinking is hard, let's go AI shopping.

so, this idea looks like follows: expose programmatic access to your program, which potentially operates in destructive manner (no Undo button) on potentially sensitive data; give a sloppy LLM (sloppy - due to its sheer unpredictability and ability to fuck up things a sober human with common sense never ever would) a Python interpreter; then let it run away with it and hope that your boundaries are enough to stop it at the edges YET don't limit the user too much?

nah, I'm skipping this update.

I'd like to see this with a proper local "instruction cache."

It might even be fun that the first call generates python (or other langauge), and then subsequent calls go through it. This "otpimized" or "compiled" natural langauge is "LLMJitted" into python. With interesting tooling, you could then click on the implementation and see the generated cod, a bit like looking at the generated asssembly. Usually you'd just write in some hybrid pytnon + natural language, but have the ability to look deeper.

I can also imagine some additional tooling that keeps track of good implementations of ideas that have been validated. This could extend to the community. Package manager. Through in TRL + web of tust and... this could be wild.

Really tricky functions that the LLM can't solve could be delegated back for human implementation.