Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Katakate – Dozens of VMs per node for safe code exec

github.com

Post reply on HN
Print view View original

1–10 of 58 posts

Show HN: Katakate – Dozens of VMs per node for safe code exec

#1

Post by gbxk »

I've built this to make it easy to host your own infra for lightweight VMs at large scale.

Intended for exec of AI-generated code, for CICD runners, or for off-chain AI DApps. Mainly to avoid Docker-in-Docker dangers and mess.

Super easy to use with CLI / Python SDK, friendly to AI engs who usually don't like to mess with VM orchestration and networking too much.

Defense-in-depth philosophy.

Would love to get feedback (and contributors: clear & exciting roadmap!), thx

Show HN: Katakate – Dozens of VMs per node for safe code exec
github.com

Re: Show HN: Katakate – Dozens of VMs per node for safe code exec

#3

Post by mentalgear »

I would really like to see a good local sandboxing solution in this space, something that is truly local-first. This is especially important since many coding models / agentic builders will eventually become lightweight enough to run them on-device instead of having to buy tokens and share user data with big LLM cloud providers.

Re: Show HN: Katakate – Dozens of VMs per node for safe code exec

#5

Post by dloss »

mentalgear wrote: post #3

I would really like to see a good local sandboxing solution in this space, something that is truly local-first. This is especially important since many coding models / agentic builders will eventually become lightweight enough to run them on-device instead of having to buy tokens and share user data with big LLM cloud providers.

Anthropic recently released a sandboxing tool based on bubblewrap (Linux, quite lightweight) and sandbox-exec (macOS). https://github.com/anthropic-experimental/sandbox-runtime

I wonder if nsjails or gVisor may be useful as well. Here's a more comprehensive list of sandboxing solutions: https://github.com/restyler/awesome-sandbox

Re: Show HN: Katakate – Dozens of VMs per node for safe code exec

#6

Post by alexeldeib »

as someone in the space this ticks a lot of boxes: kubernetes-native, strong isolation, python sdk (ideal for ML scenarios). devmapper is a nice ootb approach.

Glancing at the readme, is your business model technical support? Or what's your plan with this?

Anything interesting to share around startup time for large artifacts, scaling, passing through persistent storage (or GPUs) to these sandboxes?

Curious what things like 'Multi-node cluster capabilities for distributed workloads' mean exactly? inter-VM networking?

Re: Show HN: Katakate – Dozens of VMs per node for safe code exec

#7

Post by gbxk »

mentalgear wrote: post #3

I would really like to see a good local sandboxing solution in this space, something that is truly local-first. This is especially important since many coding models / agentic builders will eventually become lightweight enough to run them on-device instead of having to buy tokens and share user data with big LLM cloud providers.

(sorry I didn't reply in-thread, I'm new to HN, re-posting response here:)

Exactly! The main local requirement is to have hardware virtualization available (e.g. /dev/kvm) but that should be fine on your local linux machine. Won't work in cloud machines or on Mac ARM in current form but maybe if I extend

Re: Show HN: Katakate – Dozens of VMs per node for safe code exec

#8

Post by gbxk »

dloss wrote: post #5
mentalgear wrote: post #3

I would really like to see a good local sandboxing solution in this space, something that is truly local-first. This is especially important since many coding models / agentic builders will eventually become lightweight enough to run them on-device instead of having to buy tokens and share user data with big LLM cloud providers.

Anthropic recently released a sandboxing tool based on bubblewrap (Linux, quite lightweight) and sandbox-exec (macOS). https://github.com/anthropic-experimental/sandbox-runtime I wonder if nsjails or gVisor may be useful as well. Here's a more comprehensive list of sandboxing solutions: https://github.com/restyler/awesome-sandbox

wow that's super new! Thanks for that, will look deeply into it and compare

Re: Show HN: Katakate – Dozens of VMs per node for safe code exec

#9

Post by ofrzeta »

gbxk wrote: post #7
mentalgear wrote: post #3

I would really like to see a good local sandboxing solution in this space, something that is truly local-first. This is especially important since many coding models / agentic builders will eventually become lightweight enough to run them on-device instead of having to buy tokens and share user data with big LLM cloud providers.

(sorry I didn't reply in-thread, I'm new to HN, re-posting response here:) Exactly! The main local requirement is to have hardware virtualization available (e.g. /dev/kvm) but that should be fine on your local linux machine. Won't work in cloud machines or on Mac ARM in current form but maybe if I extend

There are some providers that offer KVM nested virtualization, I think Google Cloud, Digital Ocean ... any others?

Re: Show HN: Katakate – Dozens of VMs per node for safe code exec

#10

Post by gbxk »

alexeldeib wrote: post #6

as someone in the space this ticks a lot of boxes: kubernetes-native, strong isolation, python sdk (ideal for ML scenarios). devmapper is a nice ootb approach. Glancing at the readme, is your business model technical support? Or what's your plan with this? Anything interesting to share around startup time for large artifacts, scaling, passing through persistent storage (or GPUs) to these sandboxes? Curious what thing…

No business model short-term. My goal is broad adoption, 100% open-source.

By multi-node I mean so far I only support 1 k8s node, i.e. 1 machine, but soon adding support for multiple. Still, on 20 CPUs I can run +50 VM pods with fractional vCPU limits.

For GPU passthrough: not possible today because I use Firecracker as VMM. On roadmap: Add support for Qemu, then GPU passthrough possible.

Inter-VM networking: it's already possible on single-node: 1 VM = 1 pod. Can have multiple pods per node (have a look at utils/stress-test.sh). Right now I default deny-all ingress for safety (because by default k8s allows inter pod communication), but can make ingress configurable.

Startup time: a second, or a few seconds, depending on which base image (alpine, ubuntu, etc...) and whether you use a before_script or not (what I execute before the network lockdown)

Large artifacts: you can configure resource allocated to a VM pod in the sandbox config and it basically uses k8s resource limits.

Let me know if any other question! Happy to help

Post reply on HN