Great work! Though I see some people criticizing the usefulness of this. Are they being sarcastic are just really not understanding what is being discussed here? I can't tell. Maybe as an interesting follow up you could train the transformer on something with a more limited vocabulary. Spoken language is complex but a transformer can work on less complex domains like music or PET-BASIC code.
Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
41–48 of 48 posts
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#42Earlier quoted context omitted.
so... it is vibe-code? meh
Yes. The author mentions Claude for testing, but it was obviously used for the README and code as well. This is a giveaway for AI generation, from the docstring to the terrible opcode dispatch (Claude sucks at assembly or low-level optimization): https://github.com/gizmo64k/soulplayer-c64/blob/main/src/cpu... A human would use a proper dispatch table and wouldn't make excuses for a sloppy implementation ("Python is f…
I am the subject of your investigation. So, in your world meta-programming is a bad thing? Fine. In my world it isn't. The transition layers are how I kept four implementations bit-identical through the test suite. If you prefer to hand-roll this toward the goal, that's your decision, your life.
And yeah, I use AI where it makes sense. Architecture decisions are still mine. For the record: I'm from Farbrausch, so you are technically correct! The demoscene did become a UNESCO intangible cultural heritage a few years back, I guess that makes me an artist, FINALLY! :)
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#43Earlier quoted context omitted.
The Transformer is the more powerful model than Markov chain, but on such a weak machine as the C64, a MC could output text faster - but it surely would sound "psychedelic", as the memory limits a MC to a first-order or second-order model, so to predict one word, only the two words before would be taken into account as context (and no attention). On a plain vanilla C64, the Transformer cannot really show what it's ca…
You can build an unlimited-order Markov chain by, instead of pre-computing a table of counts for all possible contexts, using a substring-search index on the training data to count possible continuations on the fly: https://arxiv.org/abs/2401.17377 That paper uses suffix arrays, but more compact indices are possible: https://arxiv.org/abs/2506.12229
Could that be implemented in Hailo for Perl (it's two commands away from a base install:)
- Install Cpanminus for Perl, some C compiler and sqlite3 just to be sure.
cpanm -n local::lib
cpanm -n Hailo
~/perl5/bin/hailo -t corpus.txt -b chatbot.brn
~/perl5/bin/hailo -b chatbot.brnRe: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#44Ok now we need 1541 flash attention. I'm not sure what the venn diagram of knowledge to understand what that sentence is suggesting looks like, it's probably more crowded in the intersection than one might think.
Believe me, using the 1541 as co-processor and extra storage was super tempting and on my mind all the time! So what do you think? Flash attention with K on the front side and V on the backside? :)
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#45Ok now we need 1541 flash attention. I'm not sure what the venn diagram of knowledge to understand what that sentence is suggesting looks like, it's probably more crowded in the intersection than one might think.
Believe me, using the 1541 as co-processor and extra storage was super tempting and on my mind all the time! So what do you think? Flash attention with K on the front side and V on the backside? :)
..and we would keep the human in the loop:)
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#46Earlier quoted context omitted.
You can build an unlimited-order Markov chain by, instead of pre-computing a table of counts for all possible contexts, using a substring-search index on the training data to count possible continuations on the fly: https://arxiv.org/abs/2401.17377 That paper uses suffix arrays, but more compact indices are possible: https://arxiv.org/abs/2506.12229
Could that be implemented in Hailo for Perl (it's two commands away from a base install:) - Install Cpanminus for Perl, some C compiler and sqlite3 just to be sure. cpanm -n local::lib cpanm -n Hailo ~/perl5/bin/hailo -t corpus.txt -b chatbot.brn ~/perl5/bin/hailo -b chatbot.brn
You would probably have to completely rewrite its internals.
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#47Great work! Though I see some people criticizing the usefulness of this. Are they being sarcastic are just really not understanding what is being discussed here? I can't tell. Maybe as an interesting follow up you could train the transformer on something with a more limited vocabulary. Spoken language is complex but a transformer can work on less complex domains like music or PET-BASIC code.
Thanks! The training corpus and code are in the repo if you want to try... Training takes just a couple of minutes on an RTX 3090. Don't get your hopes up too high, though. I can imagine that code would be harder, not easier. Even modest sized transformer models struggle with proper GOTO targeting. It would look like BASIC, but essentially it would be friendly gibberish too.
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#48Dissapointed - there was no 6502 code in the GitHub repo.
I think there is, in a way: https://github.com/gizmo64k/soulplayer-c64/blob/main/src/asm...