TinyGrad is GeoHot's system/compiler to map neural networks onto hardware. He consistently points out this one point: Because the exact number of cycles is know in advance, it can be scheduled, there's no need for branch prediction, or that type of thing in a CPU. Essentially, he wants to be able to express programs, and even an operating system, as a directed acyclic graph of logical binary operations, so that you c…
> he wants to be able to express programs, and even an operating system, as a directed acyclic graph of logical binary operations, so that you can have consistent and deterministic runtime behavior. So how is this different from digital logic synthesis for CPLD/FPGA or chip design we have been doing over the last decades?
The idea is to be able to compile/run like you can now with your Von Neuman machine.
FPGA compile runs can sometimes take days! And of course, chips take months and quite a bit of money for each try through the loop.