61–70 of 173 posts
Could someone please share how such open source micro-LLMs might have been created? Do the creators take something like DeepSeek, and then delete most of the neurons to whittle down the size?
Technically, you could do that, but we trained this one from the ground up!
Please, though, take a pass at humanizing the text on the page. It's Clauded up all over and makes it hard to read.
what does the first L mean in LLM?
So the practical answer to your question is: not much.
Looking forward to npm version of needle-rs supporting v2. I added needle support for tool use in my side project.
thanks, give the playground a go and let us know how to improve!
Earlier quoted context omitted.
Technically, you could do that, but we trained this one from the ground up!
That sounds like an enormously expensive exercise.
Once you have that, the model is small enough batch sizes are probably enormous and training can probably be done on a consumer-grade GPU in a week or less. Or even faster on a bigger GPU.
{ "function_calls": [ { "name": "lock_door", "arguments": { "door": "tv" } } ], "confidence": 0.0158 }
Very interesting, seems confidence is 0 when tool calls are right?
Any instructions available for running this on an ESP32-S3 or P4 like the site says?
Hey! Roman here from Cactus - yes, we're putting putting together a detailed guide for ESP32. In the meantime, if you have enough RAM for the current model (≈28MB), our repo will get you up & running: https://github.com/cactus-compute/needle