Viewing profile — sethkim

This is extremely true. In fact, from what we see many/most of the problems to be solved with LLMs do not have ground-truth values; even hand-labeled data tends to be mostly subjec…

Feel free to shoot me a note at seth@sutro.sh if you want to check it out!

We build a product that's somewhat similar in spirit to DSPy, but people come to us for different reasons than the OP listed here. 1) It's slow: you first have to get acquainted wi…

Under-discussed superpower of LLMs is open-set labeling, which I sort of consider to be inverse classification. Instead of using a static set of pre-determined labels, you're using…

The models you called out at the beginning were all released this year. What do you think is the difference between this generation of models and previous ones?

Yes! Both Llama 3 and Gemma 3 have 128k context windows.

Yes, we're a startup! And LLM inference is a major component of what we do - more importantly, we're working on making these models accessible as analytical processing tools, so we…

My two cents here is the classic answer - it depends. If you need general "reasoning" capabilities, I see this being a strong possibility. If you need specific, factual information…

No doubt prices will continue to drop! We just don't think it will be anything like the orders-of-magnitude YoY improvements we're used to seeing. Consequently, developers shouldn'…

Both great points, but more or less speak to the same root cause - customer usage patterns are becoming more of a driver for pricing than underlying technology improvements. If so,…

I run a batch inference/LLM data processing service and we do a lot of work around cost and performance profiling of (open-weight) models. One odd disconnect that still exists in L…

Sutro.sh (fka Skysight) | Infrastructure/LLMs & Research Engineering | SF Bay Area | Full-time We are building batch inference infrastructure and a great/user developer experience …

Skysight | Infrastructure/LLMs & Research Engineering | SF Bay Area | Full-time We are building large-scale batch inference infrastructure and a great/user developer experience aro…

How "huge" are these datasets? Did you build your own tooling to accomplish this?

Thanks for the reply and the notes. On 4. specifically we've got some thoughts here as well. Will reach out!

Skysight | Infrastructure/LLMs & Product Engineering | SF Bay Area | Full-time We are building large-scale, data-intensive inference tooling and a great/user developer experience a…

What's extremely confusing to me (as a private pilot) is that traffic is almost always routed directly over an airport (midfield), to safely avoid departing and landing traffic. Th…