did you measure token/s?
Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
31–40 of 126 posts
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#32This couldn't be more perfectly timed .. I have an Unreal Engine game with both VT100 terminals (for running coding agents) and Z80 emulators, and a serial bridge that allows coding agents to program the CP/M machines: https://i.imgur.com/6TRe1NE.png Thank you for posting! It's unbelievable how someone sometimes just drops something that fits right into what you're doing. However bizarre it seems.
Connections: Alternative History of Technology by James Burke documents these "coincidences".
Those "coincidences" in Connections are really no coincidence at all, but path dependence. Breakthrough advance A is impossible or useless without prerequisites B and C and economic conditions D, but once B and C and D are in place, A becomes obvious next step.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#33There are two things happening here. A really small LLM mechanism which is useful for thinking about how the big ones work, and a reference to the well known phenomenon, commonly dismissively referred to as a "trick", in which humans want to believe. We work hard to account for what our conversational partner says. Language in use is a collective cultural construct. By this view the real question is how and why we humans understand an utterance in a particular way. Eliza, Parry, and the Chomsky bot at http://chomskybot.com work on this principle. Just sayin'.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#34As far as I know, the last layer is very quantization-sensitive, and is typically not quantized, or quantized lightly. Have you experimented with having it less quantized, and evaluated the quality drop? Regardless, very cool project.
(Not OP)
It depends on the model, but from my experiments (quantizing one layer of a model to 2-bit and then training the model with that layer in 2-bit to fix the damage) the first layer is the most sensitive, and yes, the last layer is also sensitive too. The middle layers take the best to quantization.
Different components of a layer also have a different sensitivity; e.g. the MLP downscale block damages the model the most when quantized, while quantizing the Q projection in self attention damages the model the least.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#35I love it, instant Github star.
I wrote an MLP in Fortran IV for a punched card machine from the sixties (https://github.com/dbrll/Xortran), so this really speaks to me.
The interaction is surprisingly good despite the lack of attention mechanism and the limitation of the "context" to trigrams from the last sentence.
This could have worked on 60s-era hardware and would have completely changed the world (and science fiction) back then. Great job.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#36This is super cool. Would love to see a Z80 simulator set up with these examples to play with!
100% Please do this! I wish the same
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#37Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#38This couldn't be more perfectly timed .. I have an Unreal Engine game with both VT100 terminals (for running coding agents) and Z80 emulators, and a serial bridge that allows coding agents to program the CP/M machines: https://i.imgur.com/6TRe1NE.png Thank you for posting! It's unbelievable how someone sometimes just drops something that fits right into what you're doing. However bizarre it seems.
Super intrigued but annoyingly I can’t view imgur here
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#39In before AI companies buy up all the Z80s and raise the prices to new heights.
Too late, they stopped being available last year.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#40Between this and RAM prices Zilog stock must be up! Awesome hack. Now apply the same principles to a laptop and take a megabyte or so, see what that does.