Viewing profile — rshemet

Did you give it a tool to increase temperature, or only one that sets temperature to an absolute value? Either way, setting temperature to 5° is obviously wrong - even if it knew t…

hey Kenny, Roman from Cactus here - could you say more? What kind of home assistant / what stack

it stands for Lets-not-be-sarcastic :)

Roman from Cactus here - yes you're right, there's only so much a 14MB model can do. Needle excels at in-conext inference, with tightly defined environments. In our experience: acc…

there are android binaries you can ship in your own app - https://huggingface.co/Cactus-Compute/needle2/tree/main but if you're just looking for somewhere to try the model, use our…

Hey! Roman here from Cactus - yes, we're putting putting together a detailed guide for ESP32. In the meantime, if you have enough RAM for the current model (≈28MB), our repo will g…

Hi HN, Roman and Henry here from Cactus ( https://github.com/cactus-compute/cactus ). We just shipped the biggest upgrade to our on-device inference platform: - Built-in model conf…

Yes! Cactus is optimized for mobile CPU inference and we're finishing internal testing of hybrid kernels that use the NPU, as well other chips. We don't advise using GPUs on smartp…

indeed, this is exactly the goal! The license grants rights to commercial use, unlocks additional hardware acceleration, includes cloud telemetry, and offers significant savings ov…

you can run it in Cactus Chat (download from the Play Store)

you can also run it on Cactus - either in Cactus Chat from the App/Play Store or by using the Cactus framework to integrate it into your own app

THIS IS THE BOMB!!! So excited for this one. Thanks for putting cool tech out there.

if you ever end up trying to take this in the mobile direction, consider running on-device AI with Cactus – https://cactuscompute.com/ Blazing-fast, cross-platform, and supports ne…

https://play.google.com/store/apps/details?id=com.rshemetsub...

thank you! Very kind feedback, and we'll add your feedback to our to-dos. re: "question would get stuck on the last phrase and keep repeating it without end." - that's a limitation…

say more about "community tools"?

in the app you mean? Adding shortly!

that's our mission! if you are passionate about the space, we look forward to your contributions!

no, good observation - not hidden; we don't have a "clear conversation" button. to your previous point - Cactus fully supports tool calling (for models that have been instruction-t…

looking forward to your feedback!

hot off the press in our latest feature release :) we support cloud fallback as an add-on feature. This lets us support vision and audio in addition to text.

great observation - this data is not from a controlled environment; these are metrics from our Cactus Chat use (we only collect tok/sec telemetry). S25 is an outlier that surprised…

thank you! We're continue to add performance metrics as more data comes in. A Qwen 2.5 500M will get you to ≈45tok/sec on an iPhone 13. Inference speeds are somewhat linearly inver…

Great question. Currently, each app is sandboxed - so each model file is downloaded inside each app's sandbox. We're working on enabling file sharing across multiple apps so you do…

reminds me of - "You are, undoubtedly, the worst pirate i have ever heard of" - "Ah, but you have heard of me" Yes, we are indeed a young project. Not two weeks, but a couple of mo…