interesting, i am wondering how far can it go if we remove some of these limitations but try to solve some extremely specific problem like generating regex based on user input? i know small models(270M range) can do that but can it be done in say < 10MB range?
Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
11–20 of 126 posts
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#12Imagine, this working on a Gameboy, in those days. Would've sounded like magic
I don’t think this could beat an ELIZA-style bot in how magical it feels, given the extreme terseness of its replies.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#13Imagine, this working on a Gameboy, in those days. Would've sounded like magic
I love these thought experiments. Looking at the code size, it would have been possible for someone to come up with this back in the days, similar to the idea of a million monkeys on a typewriter eventually producing Shakespeare.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#14Eliza's granddaughter.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#15Awesome. I've just designed and built my own z80 computer, though right now it has 32kb ROM and 32kb RAM. This will definitely change on the next revision so I'll be sure to try it out.
RAM is very expensive right now.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#16Nice - that will fit on a Gameboy cartridge, though bank switching might make it super terrible to run. Each bank is only 16k. You can have a bunch of them, but you can only access one bank at a time (well, technically two - bank 0 is IIRC always accessible).
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#17If one would train an actual secret (e.g. a passphrase) into such a model, that a user would need to guess by asking the right questions. Could this secret be easily reverse engineered / inferred by having access to models weights - or would it be safe to assume that one could only get to the secret by asking the right questions?
I don’t know, but your question reminds me of this paper which seems to address it on a lower level: https://arxiv.org/abs/2204.06974
“Planting Undetectable Backdoors in Machine Learning Models”
“ … On the surface, such a backdoored classifier behaves normally, but in reality, the learner maintains a mechanism for changing the classification of any input, with only a slight perturbation. Importantly, without the appropriate "backdoor key", the mechanism is hidden and cannot be detected by any computationally-bounded observer. We demonstrate two frameworks for planting undetectable backdoors, with incomparable guarantees. …”
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#18If one would train an actual secret (e.g. a passphrase) into such a model, that a user would need to guess by asking the right questions. Could this secret be easily reverse engineered / inferred by having access to models weights - or would it be safe to assume that one could only get to the secret by asking the right questions?
> this secret be easily reverse engineered / inferred by having access to models weights
It could with a network this small. More generally this falls under "interpretability."
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#19Awesome. I've just designed and built my own z80 computer, though right now it has 32kb ROM and 32kb RAM. This will definitely change on the next revision so I'll be sure to try it out.
RAM is very expensive right now.
We're talking kilobytes, not gigabytes. And it isn't DDR5 either.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#20An LLM in a .com file? Haha made my day
SLM