Viewing profile — adiraja

We're building a developer tool (currently a VSCode / Cursor extension) that helps performance optimization engineers. One of the things we hated when analyzing performance and wri…

Thank you! Absolutely, I'll send over a DM and we can take it from there!

Thanks for your comments. Absolutely, as we were mentioning in one of the other threads, we are really keen on building towards having a reproducible dashboard of efficiency and ot…

Awesome, we really appreciate the suggestions! We'll look into getting these up and running shortly!

It now makes sense that when we tested the domain ncompass.com it took us to a Microsoft home page, which is why we're ncompass.tech :)

Hey, great that you mentioned this. We actually had BAAI/bge-m3 on our list of models to put up in the near future to see if people had use for it over an API. It's great to hear t…

Per user throughput might be lower at the moment yes. We're working on GPU kernel level optimizations now to fix that. But across all users on our system, the throughput is better …

We focused mainly on the scheduling side of things. So we essentially prioritize prefills over decodes. In order to do this correctly, we had to monitor KV cache usage and whenever…

Hello HackerNews! I’m excited to share what we’ve been working on at nCompass Technologies: an AI inference* platform that gives you a scalable and reliable API to access any open-…