Earlier quoted context omitted.
We're talking kilobytes, not gigabytes. And it isn't DDR5 either.
Yeah, even an average household can afford 40k of slow DRAM if they cut down on luxuries like food and housing.
Busy cut down on the avocado toast!
Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
51–60 of 126 posts
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#52Very cool. Did you consider using sparse weights?
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#53it's giving Eliza! Ha, fun
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#54Nice - that will fit on a Gameboy cartridge, though bank switching might make it super terrible to run. Each bank is only 16k. You can have a bunch of them, but you can only access one bank at a time (well, technically two - bank 0 is IIRC always accessible).
You have 32KB of ROM, plus 8 Kb of ram on original game boy. Game boy color has more. Bank switching is super fast, as well. Given that models are likely streamed, I doubt the bank switching is a problem.
Biggest pain point is likely the text input.
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#55Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#56Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#57There are two things happening here. A really small LLM mechanism which is useful for thinking about how the big ones work, and a reference to the well known phenomenon, commonly dismissively referred to as a "trick", in which humans want to believe. We work hard to account for what our conversational partner says. Language in use is a collective cultural construct. By this view the real question is how and why we hu…
MAYBE
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#58interesting, i am wondering how far can it go if we remove some of these limitations but try to solve some extremely specific problem like generating regex based on user input? i know small models(270M range) can do that but can it be done in say < 10MB range?
Generate an LLM that is designed to solve one extremely specific problem: answering the ultimate question of life, the universe, and everything. Even with modern supercomputing the computation would be outpaced by the heat death of the universe, so token output must be limited to a single integer.
00101010
Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
#59This couldn't be more perfectly timed .. I have an Unreal Engine game with both VT100 terminals (for running coding agents) and Z80 emulators, and a serial bridge that allows coding agents to program the CP/M machines: https://i.imgur.com/6TRe1NE.png Thank you for posting! It's unbelievable how someone sometimes just drops something that fits right into what you're doing. However bizarre it seems.
Oh dear, it seems we've... somehow been psychically linked... I developed a browser-based CP/M emulator & IDE: https://lockboot.github.io/desktop/ I was going to post that instead, but wanted a 'cool demo' instead, and fell down the rabbit hole.
I've been playing the Z80-μLM demos in your CP/M emulator. Works great! However, I have yet to guess a correct answer in GUESS.COM! I'm not sure if I'm just not asking the right questions or I'm just really bad at it!