Viewing profile — adilhafeez

Is it just me or is GPU capacity surprisingly hard to find right now , even in Spot pools? I’ve been trying to get H100s and the Spot pools I’ve checked are pretty much always full…

Hey HN — I’m Adil from Katanemo (with Salman, Shuguang, and Meiyu) We previously shared an early version of this project as ArchGW. Based on customer feedback, the scope expanded f…

Hi HN — we’ve been working on a small project that routes between different LLMs based on user-defined preferences rather than just benchmarks or latency. The earlier version (Arch…

Short answer is latency impact is very minimal. We use envoy as request handler which forwards request to local service written in rust. Envoy is proven to be high performance, low…

Hi HN — we're the team behind Arch ( https://github.com/katanemo/archgw ), an open-source proxy for LLMs written in Rust. Today we're releasing Arch-Router ( https://huggingface.co…

thanks for sharing - this is quite insightful

Hi - this is Adil the co-founder who developed archgw. We are working tirelessly to create a framework that would help developers write agentic application without having to write …

Envoy has proven itself in the industry and we didn't want to reinvent the wheel by doing what envoy had already done for observability, rate-limits, connection management etc. And…

Thanks :) thanks for those comments. Those are great questions. Let me respond to them one by one, > Can I use just the model itself? yes - our models are on huggingface. You can u…

Thanks! My responses inline, > do you have to use envoyproxy to use archgw Yes, 100%. Our gateway is implemented as rust filter which runs inside envoy process. > Can archgw be use…

Thanks! Those are all good questions. Let me respond to them one by one, > Can I just use arch for routing between LLMs Yes, you can use arch_config.yaml file to select between LLM…

Hi HN! This is Adil, Salman, Co and Shuguang and we're excited to introduce archgw [1], an open source intelligent proxy for agents built on Envoy [2]. Arch moves the critical but …

You are right, since arch is an ingress wasm filter it can be setup inside Istio just like any other envoy filter. You would need to pass arch_config someone which should be easy. …

Hi HN! My name is Adil Hafeez, and I am the Co-Founder at Katanemo and the lead developer behind Arch - an open source project for developers to build faster, generative AI apps. P…

Jailbreak ensures a smooth developer experience by controlling what traffic from user make its way to the model. With jailbreak (and other guardrails soon to be added) developers c…

You can also see current list of issues at https://github.com/katanemo/arch/issues , and can also post new feature requests and bug fixes there.

Hi HN! My name is Adil Hafeez, and I am the Co-Founder at Katanemo and the lead developer behind Arch - an open source project for developers to build faster, generative AI apps. P…

Good initiative

pretty cool