Viewing profile — vrm

good luck running a 2.4T model on any local hardware. it’s not gonna happen. the arrow is to specialized hardware at least for the smartest models

TBH it's intended only for internal use (we don't even publish it as a crate at this point) so I don't particularly mind it being low-key. But I appreciate it!

If you don't need a ton of throughput I think `absurd` (and our Rust derivative `durable`) are very nice options that keep the client side extremely simple. It's also lightweight e…

Since DBOS doesn't support Rust, we implemented a very minimal Rust version of this at https://github.com/tensorzero/durable . It has been quite stable and extensible but of course…

https://twitter.com/arnal_charles/status/2060009395107377282 , https://xcancel.com/arnal_charles/status/2060009395107377282 Paper: Formalizing Mathematics at Scale - https://arxiv.…

It’s really not a concept you can express in idiomatic Python very easily. This comes from the actual generated assembly involving copies from global GPU memory into registers (slo…

One question I have here: I think this type of thing would be trivial to do in Rust with constructors, private fields, and newtypes. What am I getting on top of it?

I maintain an OSS project with a very involved CI setup. We're at the point where it is worth having observability into which tests are flaky, especially within intra-test-run retr…

that is earnings (net income) not revenue (top line) so these are wildly different and incomparable numbers

a 6:1 parameter ratio is too small for specdec to have that much of an effect. You'd really want to see 10:1 or even more for this to start to matter

This is neat! I think in general there are really deep connections between semantically meaningful diffs (across modalities) and supervision of AI models. You might imagine a human…

if you haven't check out our repo -- it's free, fully self-hosted, production-grade, and designed for precisely this application :) https://github.com/TensorZero/tensorzero

I definitely see different prompts based on what I'm doing in the app. As we mentioned there are different prompts for if you're asking questions, doing Cmd-K edits, working in the…

we're doing the latter! Cursor lets you configure the OpenAI base URL so we were able to have Cursor call Ngrok -> Nginx (for auth) -> TensorZero -> LLMs. We explain in detail in t…

wireshark would work for seeing the requests from the desktop app to Cursor’s servers (which make the actual LLM requests). But if you’re interested in what the actual requests to …

We're working on an OSS industrial-grade version of this at TensorZero but there's a long way to go. I think the easiest out of the box solution today is probably OpenAI RFT but th…

This is very neat work! Will be interested in how they make this sort of thing available to the public but it is clear from some of the results they mention that search + LLM is on…

would it be possible to fuzz an arbitrary JSON schema with this? I’ve been looking for such a lib for a while now

OP here: I saw the spinlaunch video and got really excited that this was completely tractable in simplified form with basic physics. So I did the math expecting to see a huge savin…