[deleted]
An SLM trained on $8 ESP32-S3
11–20 of 23 posts
Re: An SLM trained on $8 ESP32-S3
#12Very cool project. > Backpropagation (gradients derived by hand) What does by hand mean in this context? Also how did you write the readme? It's a curious blend of human and AI writing.
Author here. Fair question, and the answer is in the first line of the README: I wrote it, with AI help.
Specifically: I wrote it in Spanish first, which is my language, and then translated and condensed it into English with Claude. That's probably the blend you're picking up. The jokes, the structure and the decisions are mine; the English phrasing has fingerprints that aren't.
The code is more clear cut and it says so in the header: generated by AI under my direction. I wrote the architecture, the decisions and the validation, not most of the C.
I'd rather say that upfront than have someone find it later.
Re: An SLM trained on $8 ESP32-S3
#13Very cool project. > Backpropagation (gradients derived by hand) What does by hand mean in this context? Also how did you write the readme? It's a curious blend of human and AI writing.
The fact that by hand is emphasized so often and so often (see src/handgpt.h comments as well above backward()) makes me think it's AI. Anyways, it looks like gradients derived by hand means they didn't use autograd. They have written out the expression for dL/dW themselves.
Correct. No autograd: the expressions for the gradients are written out explicitly in C. There's a gradient check in tests/ that verifies them against centred finite differences on the published header, worst relative error 1.07e-08.
Re: An SLM trained on $8 ESP32-S3
#14> Which is no small thing.
I think it is a small thing.
Re: An SLM trained on $8 ESP32-S3
#15[flagged]
Re: An SLM trained on $8 ESP32-S3
#16[flagged]
I understand that you are alluring that maximum training-state memory, not parameter count, is the key variable here. So you better start with the smallest model, and go to for highest training data quality, with the outlook of coupling systems together?
Re: An SLM trained on $8 ESP32-S3
#17Would probably be better to demonstrate by exsmple how this approach is used to train on sensor data and then use it (as is hinted by the author) instead of acknowledging that the klingon poc is useless.
But would it be as cool as a microcontroller that spits out Klingon?
Re: An SLM trained on $8 ESP32-S3
#18hIngan motlh puS ruq tuq DujDaq SISwI' nge'vI' vIn SuvwI' yIvwI' qarghtaHvIS SIchoH
vIparHa'bej.
Re: An SLM trained on $8 ESP32-S3
#19Awesome.
Re: An SLM trained on $8 ESP32-S3
#20[flagged]
I understand that you are alluring that maximum training-state memory, not parameter count, is the key variable here. So you better start with the smallest model, and go to for highest training data quality, with the outlook of coupling systems together?
You're talking to an LLM, unfortunately
https://news.ycombinator.com/threads?id=runtime_lens
(enable showdead in your HN settings)