Viewing profile — gskm

Thanks! Glad it's working well for you. A few practical tips: 1. Pass the user's query directly. In the benchmark, the hint is literally the question. That's the simplest and most …

Update on the benchmark numbers: the results in the original post were computed with a looser tokenizer, making the budget less strict than it should be. We've since improved that …

Thank you! That's exactly the goal, drop-in token savings without changing your LLM pipeline. If you give it a spin, I'd love to hear how it works on your data. We're actively tuni…

[dead]

Loved this article. I'd add a few things I wish someone had told me when I was starting my PhD: 1) Maximize variance, but know when to stop. Karpathy's point is great. Explore earl…

[flagged]

I think your instinct is right. More context isn't free, even when the window supports it, and the model still has to attend to everything in there, and noise dilutes the signal. A…

The context-overload rule resonates — we kept hitting the same problem. Diagnosis is useful but we ended up just compressing the retrieved chunks to a token budget before they hit …