Viewing profile — roh26it

What do you use for the onboarding guide?

Well written, but does this all change in a claude-ified world?

Hi HN! I'm Rohit, co-founder and CEO of Portkey. After building our open-source AI gateway that now handles billions of tokens daily, we kept seeing the same problem: prompt engine…

I love playing around with these tools for quick prototyping and learning about some stuff, but don't see myself using these for serious apps. Any tips on what I could do?

Looks solid, going to try it out. I'm going to be __that guy__, but just ask - is the functionality set similar to llamaparse or is this llamaparse + llm?

So, while time to first token is lower, throughput might also be lower in most cases?

What are the trade-offs you've made to achieve this?

The most interesting findings (at least from my perspective) - OpenAI measured "honesty" that could become an interesting model eval others should adopt. - Using instruction heirar…

I've been thinking about AI guardrails in production for almost 7 months now, and I'm convinced that using them to "control", "secure", or "block" LLM outputs is not the best use f…

Also wondering if this would become expensive very fast if it ends up using S3 with a large number of PUT calls

thanks for the support!

We use a bunch of caching mechanisms on the LLM requests themselves and extend the same to guardrails now. So there's 2 levels of cache - the LLM request itself might be cached (si…

Hi HN, I've been developing Portkey Gateway, an open-source AI gateway that's now processing billions of tokens daily across 200+ LLMs. Today, we're launching a significant update:…

Being one of the most downloaded datasets on Huggingface, I was a little bit surprised by how dirty this dataset was. Plus it had very limited information and some incorrect classi…

Here's a mega guide on keeping costs low with LLMs - https://portkey.ai/blog/implementing-frugalgpt-smarter-llm-u... tl;dr: - Keep prompts short, combine prompts or make more detai…