Viewing profile — cafaxo

Yes, sorry -- I did not realize that for some reason. I removed my comment.

[Comment removed by author]

Does gradient descent really do well for deep learning when the gradient is computed with respect to the whole dataset? I assumed that the noise in SGD played an important role for…

I did a similar thing for Julia: Llama2.jl contains vanilla Julia code [1] for training small Llama2-style models on the CPU. [1] https://github.com/cafaxo/Llama2.jl/tree/master/sr…

To me, the clearest way to think about signed vs. unsigned integers is that different representatives of the integers modulo n are chosen. For example, for 8-bit signed integers we…

Yes, exactly.

Of course. Theoretically, the determinant answers the binary question "singular" or "nonsingular". Numerically, such a binary answer is pretty useless. Here, we need a measure of h…

Edit: Sorry, I completely messed up my original answer here. A better version: Let's say we are in a setting where we only work with integers. A matrix is invertible iff its determ…

They define a generic "is_singular" function and test it with a 2x2 matrix. The problem with the determinant is not about performance. It is just useless for determining if a matri…

From page 6, when they describe their synthetic textbook dataset: "Consider the matrix A = np.array([[1, 2], [2, 4]]). We can check if this matrix is singular or nonsingular using …

GPT-4's explanation of its optimization does not make sense to me. It writes "Instead of moving it to P, we can directly use S in the following comparisons, saving one instruction.…