Viewing profile — forrestp

My understanding is that your levers are roughly better / more diverse embeddings or computing more embeddings (embed chunks / groups / etc) + aggregating more cosine similarities …

It's expensive in this field to verify other people's work. There are a few other papers in the last 3 years that have the same high-level idea but call the anchor tokens something…

Gradient.ai | SF Bay Area | Onsite/hybrid | Staff SWE | Senior SWE | Enterprise Account Executive Our vision is to power the future of enterprise automation. Gradient is a full sta…

Right. We are sleep deprived -- couldn't stop over the weekend. Please forgive the typos

All (training / evals / inference) performed on their L40s clusters. These machines are underrated but capable of serious work

We are training on top of llama 3. The 256k reasoning benchmarks are on the open LLM leaderboard. And re: token count: our copy was wrong -- it's pre-prepped copy for a model run t…

Our team @ https://gradient.ai/ has more checkpoints coming soon with longer context lengths.