Viewing profile — fovc

Łukasz Kaiser basically confirmed it in a podcast: https://youtu.be/3K-R4yVjJfU?si=JdVyYOlxUbEcvEEo&t=2624 > Q: Are the releases aligned with pre-training efforts? > A: There used …

> I also think it’s important to notice that a lot of these challenges they happen with humans too. The concept of prompt injection isn’t that different from social engineering, ri…

Crossover from the other front page article. I tested out ChatGPT5 search mode and there are some good sources! https://chatgpt.com/s/t_68bd82908c0c8191b142b860ff91c9dc

Very nice! For other readers, vc is short for verification condition and wp is short for weakest precondition.

I wonder if the error propagation problem could be solved with a “branching” generator? Basically at every token you fork off N new streams, with some tree pruning policy to avoid …

Or see the explanation in video form here: https://m.youtube.com/watch?v=d0HJvGSWw8A Mamba has been discussed a lot here, and this seems like a promising line of inquiry for improv…

And also make sure that slot is a symbol in the correct package. Or do like Elisp and do without packages but then have a 16 character prefix

Having the data structures is nice and all, but using them is kind of painful. They are certainly second class. Having to use accessor functions or destructuring macros instead of …

Sparse attention essentially combines 3 types of attention optimizations: 1. Compression of the query input vectors to reduce the size of the KV cache 2. Selectively computing unco…

Ah thanks for the clarification! Do you happen to know if Nile/Gezira went anywhere?

Great to see this is alive and progressing! I believe Ohm started life in Alan Kay’s research group, to build a graphical OS and office suite in 10k lines of code. I found this tal…

> I feel like I'm taking crazy pills when I read about others' experiences. Surely I am not alone? You're not alone :-) I asked a very similar question about a month ago: https://n…

The “pair programming” approach with good models is just slow enough that I lose focus on each step. The faster models I’ve tried are not good enough except for straightforward thi…

I still don’t understand how people are getting value out of AI coders. I’ve tried really hard and the commits produced are just a step up from garbage. Writing code from scratch i…

Makes a ton of sense! Is this for completions, patches, or new files?

Maybe this? https://www.cis.upenn.edu/~bcpierce/tapl/

Agree it’s not a toy. AWS implemented a large chunk of IAM in Dafny. Though IIRC they have their own non-public compiler to Java

It’s also useful in typed languages to introduce an existentially quantified type