Funny how pydantic is used to parse and not validate but then there are post conditions after parsing which you should parse actually or which can be enforced with json schema and properly implemented constrained sampling on the LLM side.
Software 3.1? – AI Functions
41–50 of 61 posts
Re: Software 3.1? – AI Functions
#42I wrote about something along these lines 3 years ago, but used the name "Heisenfunctions," which I think is better :)
https://kylekukshtel.com/incremental-determinism-heisenfunct...
A lot of this was also inspired by Ian Bicking's work here:
Re: Software 3.1? – AI Functions
#43The "Grace" language is based on the same idea, but lets you get the full benefit of specifying static types.
https://github.com/Gabriella439/grace
It's still probably not a great idea.
Re: Software 3.1? – AI Functions
#44I'd like to see this with a proper local "instruction cache." It might even be fun that the first call generates python (or other langauge), and then subsequent calls go through it. This "otpimized" or "compiled" natural langauge is "LLMJitted" into python. With interesting tooling, you could then click on the implementation and see the generated cod, a bit like looking at the generated asssembly. Usually you'd just…
Nice! I can almost see your vision. In terms of tooling, I think this could be integrated with deep instrumentation (a-la datadog) and used to create self-improving systems.
Re: Software 3.1? – AI Functions
#45This is why RAM is 5x.
Re: Software 3.1? – AI Functions
#46Why stop there? Just call the LLM with the data and function description and get it to return the result! (I'll admit that I've built a few "applications" exploring interaction descriptions with our Design team that do exactly this - but they were design explorations that, in effect, used the LLM to simulate a back-end. Glorious, but not shippable.)
That's basically how it works! (with human authored functions that validate the result, automatically providing feedback to the LLM if needed)
Re: Software 3.1? – AI Functions
#47Why stop there? Just call the LLM with the data and function description and get it to return the result! (I'll admit that I've built a few "applications" exploring interaction descriptions with our Design team that do exactly this - but they were design explorations that, in effect, used the LLM to simulate a back-end. Glorious, but not shippable.)
Because you often need the result not as a standalone artifact, but as a piece in a rigid process, consisting with well-defined business logic and control flow, with which you can't trust AI yet.
Re: Software 3.1? – AI Functions
#48This has big "let's do this because we can" energy. What is the BENEFIT of all this? Let's use Blockchain instead of a database - because we can. Let's create a maze of microservices - because we can. Let's make every function a lambda function - because we can. Let's make AI write code, run it, verify it, fix it, then run it again - because we can. Let's burn untold amounts of energy to do simple things - because we…
To you, what's the point of spending countless billions on space exploration?
Re: Software 3.1? – AI Functions
#49This has big "let's do this because we can" energy. What is the BENEFIT of all this? Let's use Blockchain instead of a database - because we can. Let's create a maze of microservices - because we can. Let's make every function a lambda function - because we can. Let's make AI write code, run it, verify it, fix it, then run it again - because we can. Let's burn untold amounts of energy to do simple things - because we…
To you, what's the point of spending countless billions on space exploration?
You can make that argument about every single thing that is wasteful but can be justified as "research".
Sure, every bit of f--ing around is research, but ROI is far from constant.
Re: Software 3.1? – AI Functions
#50I'd like to see this with a proper local "instruction cache." It might even be fun that the first call generates python (or other langauge), and then subsequent calls go through it. This "otpimized" or "compiled" natural langauge is "LLMJitted" into python. With interesting tooling, you could then click on the implementation and see the generated cod, a bit like looking at the generated asssembly. Usually you'd just…
[dead]
I'm wondering if the post-condition checks change the perspective on this at all, because yes the code is nondeterministic and may execute differently each time. That is the problem this is trying to solve. You define these validation rules and they are deterministic post-condition checks that retry until the validation passes (up to a max retry number). So even if the model changes, and the behavior of that model changes, the post-condition checks should theoretically catch that drift and correct the behavior until it fits the required output.