An LLM in a .com file? Haha made my day
SLM
All the 'Small' language models and the 'TinyML' scene in general tend to bottom out at a million parameters, hence I though 'micro' is more apt at ~150k params.
Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
21–30 of 126 posts
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#22interesting, i am wondering how far can it go if we remove some of these limitations but try to solve some extremely specific problem like generating regex based on user input? i know small models(270M range) can do that but can it be done in say < 10MB range?
Generate an LLM that is designed to solve one extremely specific problem: answering the ultimate question of life, the universe, and everything.
Even with modern supercomputing the computation would be outpaced by the heat death of the universe, so token output must be limited to a single integer.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#23This couldn't be more perfectly timed .. I have an Unreal Engine game with both VT100 terminals (for running coding agents) and Z80 emulators, and a serial bridge that allows coding agents to program the CP/M machines: https://i.imgur.com/6TRe1NE.png Thank you for posting! It's unbelievable how someone sometimes just drops something that fits right into what you're doing. However bizarre it seems.
Oh dear, it seems we've... somehow been psychically linked...
I developed a browser-based CP/M emulator & IDE: https://lockboot.github.io/desktop/
I was going to post that instead, but wanted a 'cool demo' instead, and fell down the rabbit hole.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#24As far as I know, the last layer is very quantization-sensitive, and is typically not quantized, or quantized lightly.
Have you experimented with having it less quantized, and evaluated the quality drop?
Regardless, very cool project.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#25Meanwhile, Eliza was ported to BASIC and was run on many home computers in the 80s.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#26Luckily I have a very large amount of MSX computers, zx, amstrad cpc etc and even one multiprocessor z80 cp/m machine for the real power. Wonder how gnarly this is going to perform with bankswitching though. Probably not good.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#27It's pretty obvious this is just a stress test for compressing and running LLMs. It doesn't have much practical use right now, but it shows us that IoT devices are gonna have built-in LLMs really soon. It's a huge leap in intelligence—kind of like the jump from apes to humans. That is seriously cool.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#28This couldn't be more perfectly timed .. I have an Unreal Engine game with both VT100 terminals (for running coding agents) and Z80 emulators, and a serial bridge that allows coding agents to program the CP/M machines: https://i.imgur.com/6TRe1NE.png Thank you for posting! It's unbelievable how someone sometimes just drops something that fits right into what you're doing. However bizarre it seems.
Connections: Alternative History of Technology by James Burke documents these "coincidences".
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#29Pretty cool! I wish free-input RPGs of old had fuzzy matchers. They worked by exact keyword matching and it was awkward. I think the last game of that kind (where you could input arbitrary text when talking to NPCs) was probably Wizardry 8 (2001).
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#30It's pretty obvious this is just a stress test for compressing and running LLMs. It doesn't have much practical use right now, but it shows us that IoT devices are gonna have built-in LLMs really soon. It's a huge leap in intelligence—kind of like the jump from apes to humans. That is seriously cool.
i'll echo that practicality only surfaces once it is apparent what can be done. yea this feels like running "DOOM on pregnancy test devices" type of moment