Viewing profile — emil_sorensen

happy it helped!

it's a cost/latency trade-off in production + very use-case dependent

Thanks! Yep fixed

Accuracy drops hard with context length still. Especially in more technical domains. Plus latency and cost.

Docs bots like these are deceptively hard to get right in production. Retrieval is super sensitive to how you chunk/parse documentation and how you end up structuring documentation…

Thanks for the feedback. We should definitely add that. :)

OP here. It's kind of ironic that making the docs AI-friendly essentially just ends up being what good documentation is in the first place (explicit context and hierarchy, self-con…

We focus mainly on external use cases (e.g., helping companies like Docker and Monday.com deploy customer facing "Ask AI" assistants) so we don't run into much of that given all da…

Super cool! Yep, a lot seems to get lost through distillation.

I suspect you're right here! Excited to get our hands on the non-distilled o3. :)

Yep even with a small bump in performance (which we only saw for a subset of coding questions), it wouldn't be worth the huge latency penalty. Though that will surely go down over …

Curious if anyone else has run similar experiments?

That's a great point. Reminds me of the "feature, not a bug" Karpathy tweet [0]. [0]: https://x.com/karpathy/status/1733299213503787018?lang=en

I find it so interesting that it's possible to develop a "feeling" of a new model.