Viewing profile — ansk

I can see how LLMs may make compiler development seem futile when it comes to optimizing the local graph structure (ie kernel-sized chunks of the computation). But optimizing the g…

This kind of sounds like just giving up on making the compiler better.

I skimmed the doc and my impression is that your second listed interpretation -- DeepSeek is pausing investment because of a leak -- is the more correct one. There's quite a bit of…

I've found breathing exercises to be effective for the duration of the exercise, but I'm more interested in the possibility of training myself to adjust my respiration patterns ove…

Kind of a backwards take, both in your interpretation of this announcement and the company in general. The point of this announcement is to draw attention to the fact that the curr…

I can assure you that a fully deterministic and equally effective claude is possible to build. And yes, that would mean identical prompts would yield 100% identical output 100% of …

I see what you're getting at, but determinism isn't the right word either. LLMs are fundamentally deterministic -- they are pure functions which output text as a function of the in…

The guy writing a thumbnail pipeline isn't getting petabytes (exabytes?) of storage to cache all videos from the past week in their entirety. If this quantity of data is being stor…

The other explanations here don't explain the long delay between the start of the investigation and the release of the footage. Yes, storing customer data is what we'd expect from …

The implication that OpenAI is a YC company in the same sense as the other listed companies is somewhere between misleading and dishonest. Even more distasteful to show founding te…

Of all Schmidhuber's credit-attribution grievances, this is the one I am most sympathetic to. I think if he spent less time remarking on how other people didn't actually invent thi…

I can only imagine what the Taiwanese can do in Arizona. Truly a synergy for the ages.

My personal experience is that the cost of enduring a negative stimulus is not simply a function of the magnitude of the negative stimulus, but rather the magnitude of the negative…

For future reference, if you want proper python bindings for ffmpeg* you should use pyav. * To be more precise, these are bindings for the libav* libraries that underlie ffmpeg

And on the seventh day, God ended His work which He had done and began vibe coding the remainder of the human genome.

Key-based attention is not attributable to the Transformer paper. First paper I can find where keys, queries, and values are distinct matrices is https://arxiv.org/abs/1703.03906 ,…

The scientific impact of the transformer paper is large, but in my opinion the novelty is vastly overstated. The primary novelty is adapting the (already existing) dot-product atte…

Tensorflow is a lot like IBM -- it deserves praise not because it's great in its current state, but for its contributions towards advancing the broader technological front to where…

I've seen and ignored a lot of "pytorch good, tensorflow bad" takes in my time, but this is so egregiously wrong I can't help but chime in. Facilitating graph-level optimizations h…

I think that while researchers would agree that the empirical success of deep learning has been remarkable, they would still agree that the language used here -- "an algorithm that…

> humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data) He's hand-wavin…

Imagine reading a math or programming textbook where each statement was true with probability 0.95.

Is batched inference for LLMs memory bound? My understanding is that sufficiently large batched matmuls will be compute bound and flash attention has mostly removed the memory bott…

Furthermore, each of those 16 channels would typically be mutibyte floats as opposed to single byte RGB channels. (speaking generally, haven't read the paper)

Apparently my innuendo has not been taken well, so I'll clarify in more straightforward terms: @abidlabs is the employer of the individual running the aforementioned twitter accoun…