31–40 of 188 posts
Was really cool to see yous use Engrams to cut down compute! Given its basically an O(1) lookup with disk space being the main constraint, I was curious if you've tried ablating engram layers and sizes across your setup? Also, why mHC over attention residuals?
Yes, we ablated Engrams rigorously and found that it returned world knowledge like FFN without without compute expenditure.
Naïve and clumsy question: how would you pair this with speech-text-speech stuff, wake words etc.? Are there good examples of this for a Pi 5? The demo is super — I'm just having trouble seeing the whole picture for e.g. a screenless device. ETA: pun not intended
The best entrypoint is Home Assistant: https://www.home-assistant.io/ That will get you a lot further than what you're asking, but if you dig a bit through Home Assistant features, resources, etc., you may find the current "best" answers to your questions. If you want a quick answer: Whisper is a good open-source speech-to-text model which comes in a variety of sizes ( https://huggingface.co/openai/whisper-tiny ). Yo…
This is cool. I definitely think the "micro" sized LLM space is underappreciated, so it's always good to see work like this. I foresee a paradigm in some contexts where you have a hierarchy of LLMs, with more competent models actively training smaller models to solve specific tasks very efficiently, and something like this could be the smallest layer in that stack. With that being said, the web demo is not particular…
Earlier quoted context omitted.
The website says the model is for "tool calling, device use, and structured extraction". Your example just doesn't seem to be very relevant. FWIW, it did a pretty good job for tool calling when I tried it, and I think it could be pretty nice to have this running on locally and integrate with Home Assistant.
False positives are definitely relevant and worth measuring - natural language interfaces always have a discoverability problem, i.e., users not knowing what actions the system does and does not support. If the frontend of that system lacks the ability to reject unsupported commands, weird stuff happens. Nonetheless, this is very cool work! If I can offer a small suggestion to the team at Cactus, it would be to evalu…
Naïve and clumsy question: how would you pair this with speech-text-speech stuff, wake words etc.? Are there good examples of this for a Pi 5? The demo is super — I'm just having trouble seeing the whole picture for e.g. a screenless device. ETA: pun not intended
Users often stack a transcription model on top to get the voice prompt, then decode to actions. Think of Alexa and Siri.
The learned confidence gate is the crucial piece for a 14MB action model. On ambiguous requests such as the HN example, what calibration target decides between abstaining locally and escalating to the cloud?
Edit: I have a pile of d1 minis, but not much time.
import needle
@needle.tool
def add(a: int, b: int):
"Add two numbers."
return a + b
agent = needle.Needle(tools=[add])
print(agent.run("calculate 1 + 1?")["reasoning"])
conclusion: completly useless
Do the creators take something like DeepSeek, and then delete most of the neurons to whittle down the size?
Makes me think of the demo from some time ago where someone got a ~29M parameter model running on an esp32. I wonder what kind of throughput this could get if a handful of esp32s were strung together... Edit: I have a pile of d1 minis, but not much time.