Live data from Hacker News

Forcing Flash Attention onto a TPU and Learning the Hard Way

archerzhang.me

11–17 of 17 posts

Re: Forcing Flash Attention onto a TPU and Learning the Hard Way

#11
Interesting read! One remark though: I'm not too familiar with the architecture of a Google TPU, but comparing the TPU's VMEM with Nvidia's shared memory feels wrong to me.

Looking at the size, and its shared nature, it feels far more natural to compare with the L2 cache, which is also shared across the entire GPU and is in the same order of size (40MB on the listed A100).

Re: Forcing Flash Attention onto a TPU and Learning the Hard Way

#12
post #6
post #5

Earlier quoted context omitted.

I personally don't mind letting Claude write about work. You could spend 80% doing the work and 20% writing about it, or 99% doing the work and 1% copy-pasting Claude's writeup about it into a blog. There is nothing wrong with writing if you are into it, and yes you can probably do better than Claude, but I can related to engineers who just want to build.

If you can’t be bothered to write it, why should I bother to read it?

Because it contains information of value to you ? I mean if it doesn’t, just don’t read it.

Re: Forcing Flash Attention onto a TPU and Learning the Hard Way

#13
post #9

Why ruin good work by letting Claude write it all? Full of em dashes, riddled with Claudisms.

Why let an obsession with writing style prevent you from learning from a reasonably decent writeup?

He’s doing the author a favor

Re: Forcing Flash Attention onto a TPU and Learning the Hard Way

#14

Why ruin good work by letting Claude write it all? Full of em dashes, riddled with Claudisms.

i did a quick scroll and was happy to see a long-ish article on XLA and TPUs. then i realized it was literally just "using vmap for parallel loops is better than fori", but in massively wordy claudisms.

Re: Forcing Flash Attention onto a TPU and Learning the Hard Way

#15
post #6

Earlier quoted context omitted.

If you can’t be bothered to write it, why should I bother to read it?

Because it contains information of value to you ? I mean if it doesn’t, just don’t read it.

https://marketoonist.com/wp-content/uploads/2023/03/230327.n...

Re: Forcing Flash Attention onto a TPU and Learning the Hard Way

#16
post #6

Earlier quoted context omitted.

If you can’t be bothered to write it, why should I bother to read it?

Because it contains information of value to you ? I mean if it doesn’t, just don’t read it.

To quote another HN comment recently made:

> Using AI to write content is seen so harshly because it violates the previously held social contract that it takes more effort to write messages than to read messages. If a person goes through the trouble of thinking out and writing an argument or message, then reading is a sufficient donation of time.

However, with the recent chat based AI models, this agreement has been turned around. It is now easier to get a written message than to read it. Reading it now takes more effort. If a person is not going to take the time to express messages based on their own thoughts, then they do not have sufficient respect for the reader, and their comments can be dismissed for that reason.

Re: Forcing Flash Attention onto a TPU and Learning the Hard Way

#17
post #16

Earlier quoted context omitted.

Because it contains information of value to you ? I mean if it doesn’t, just don’t read it.

To quote another HN comment recently made: > Using AI to write content is seen so harshly because it violates the previously held social contract that it takes more effort to write messages than to read messages. If a person goes through the trouble of thinking out and writing an argument or message, then reading is a sufficient donation of time. However, with the recent chat based AI models, this agreement has been…

So to a large extent I appreciate that argument, however I feel this applied more to throwaway comments or sales outreach, writing with low information density. In this occasion the work that went into it is a lot, it would be lost or inaccessible to me otherwise, I am genuinely grateful someone stuck their work in an LLM, said tidy this up to post, and hit enter.
Post reply on HN