I love these counterfactual creations on old hardware. It highlights the magical freedom of creativity of software.
Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
31–40 of 48 posts
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#32Ok now we need 1541 flash attention. I'm not sure what the venn diagram of knowledge to understand what that sentence is suggesting looks like, it's probably more crowded in the intersection than one might think.
How many 40+ AI pillers? Assume 10M devs in the world. 10% heard of flash attention, 1% heard of 1541 then 10,000
Ahh but you also have to know the significance of the 1541 that makes the Flash attention reference work
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#33A little disappointed to see PyTorch + Claude here. I was hoping for some "demo-scene" hand-crafted 6502 assembly, and hopefully training on the C64.
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#34A little disappointed to see PyTorch + Claude here. I was hoping for some "demo-scene" hand-crafted 6502 assembly, and hopefully training on the C64.
so... it is vibe-code?
meh
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#35You can chat with the model on the project page: https://indiepixel.de/meful/index.html It (v3) mostly only says hello and bye, but I guess for 25k parameters you can't complain. (I think the rather exuberant copy is probably the product of Claude et al.)
It seems to like getting its own output fed back:
YOU> HI! HOW ARE YOU DOING?
C64> BELON
YOU> WHAT DO YOU MEAN BY THAT?
C64> YOU DES
YOU> YOU TOO!
C64> YOU DESER! THIS THE SHIN
YOU> DESER? SHIN? THOSE AREN'T EVEN REAL WORDS, WHAT'S GOING ON HERE?
C64>
YOU> THIS THE SHIN
C64> YOU MAKE ME HAPPY TOO
YOU> YOU DESER
C64> YOU MADE MY HEART! W
YOU> DES, DESER, DESERST?
C64> YOU DES
YOU> BELON
C64> I LIKE OURON ME! WONDERFUL!YOU!
YOU> YOU BELON DES
C64> YOU MAKE ME HAPPY
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#36Earlier quoted context omitted.
How does it compare to a Markov chain generator I wonder.
The Transformer is the more powerful model than Markov chain, but on such a weak machine as the C64, a MC could output text faster - but it surely would sound "psychedelic", as the memory limits a MC to a first-order or second-order model, so to predict one word, only the two words before would be taken into account as context (and no attention). On a plain vanilla C64, the Transformer cannot really show what it's ca…
You can build an unlimited-order Markov chain by, instead of pre-computing a table of counts for all possible contexts, using a substring-search index on the training data to count possible continuations on the fly: https://arxiv.org/abs/2401.17377 That paper uses suffix arrays, but more compact indices are possible: https://arxiv.org/abs/2506.12229
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#37A little disappointed to see PyTorch + Claude here. I was hoping for some "demo-scene" hand-crafted 6502 assembly, and hopefully training on the C64.
so... it is vibe-code? meh
Yes. The author mentions Claude for testing, but it was obviously used for the README and code as well.
This is a giveaway for AI generation, from the docstring to the terrible opcode dispatch (Claude sucks at assembly or low-level optimization): https://github.com/gizmo64k/soulplayer-c64/blob/main/src/cpu...
A human would use a proper dispatch table and wouldn't make excuses for a sloppy implementation ("Python is fast enough").
Besides, the author has an art and design background, which doesn't seem to match the deep knowledge of Transformers or assembly required for such a project.
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#38A little disappointed to see PyTorch + Claude here. I was hoping for some "demo-scene" hand-crafted 6502 assembly, and hopefully training on the C64.
Same, however I do conceed having the whole assembler toolchain written in Python was also kind of cool, even if it may have been AI generated.
Even cooler would have been to have the 6502 directly generated from the LLM.
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#39Maybe impressive in one way, but I'm also pretty sure a simple n-gram Markov model (a la Niall on the Amiga) would have a lower loss on the test set.
Transformers don't scale down very well, in my experience - I used to train local models all the time as new ones were released, as I recall transformers were the first ones I couldn't get better results out of with my limited training data and GPU.
Re: Soul Player C64 – A real transformer running on a 1 MHz Commodore 64
#40Earlier quoted context omitted.
How fast is the “new” Commodore 64? Have not heard much about it since launch. Although, now that I look, it seems they are just shipping now. https://www.commodore.net/product-page/commodore-64-ultimate...
RAM can be increased to 16 MB and CPU speed to 48 GHz.
The 64Ultimate goes to 64MHz, the Ultimate64 cartridge goes to 48MHz "only".