Viewing profile — alexandercheema

lol bro there is already a big OSS project called exo: https://github.com/exo-explore/exo

Isn't Claude Code for Infrastructure just...Claude Code?

Is there one for Kimi K2.5?

Appreciate you checking back so often. We have some exciting plans. Keep checking and it won't be long before something pops up :)

Yes, these models are mostly compute-bound so benefit even more from the compute on the DGX Spark.

Blog author here. Actually, no. The model can be streamed into the DGX Spark, so we can run prefill of models much larger than 128GB (e.g. DeepSeek R1) on the DGX Spark. This featu…

exo maintainer here. tgtweak is correct. This looks like potentially some promising research that I'm looking into reproducing now. We want to lower the barrier to running large mo…

Not yet, should I make an issue for it?

This is fixed now, with these commits: - https://github.com/exo-explore/exo/commit/dbbc7be57fb1871d2b... - https://github.com/exo-explore/exo/commit/ce46f000591d8d59c1... Please ke…

Do you mean with Apple Intelligence? You can already query models you host from Apple using exo or even just local on-device inference.

Try it out - don't trust me! The way this works is that each device holds a partition of the model (for now a continuous set of layers). E.g. let's say you have 3 devices and the m…

Repo maintainer here. It supports any device tinygrad does, which is a lot. We didn’t expect it to blow up so soon - the repo is still experimental. Internally we’ve mostly been te…

Thank you for the support! I agree on the cost point, and personally I don’t want to live in a world where all AI runs on H100s in a giant datacenter controlled by one company.

You’re right. The assumption right now is that you’re running on trusted devices on your own local network. I will add a section in the README.

That’s where we want to get eventually. There’s a lot of work that needs to be done but I’m confident we’ll get there. Give us 3 months and it’ll be as simple as running Dropbox.

Could you create a GitHub issue? There's a lot of work we'd like to do to improve this.