Earlier quoted context omitted.
You seem to be conflating "RGB" with one particular RGB color space: sRGB. That's a common enough conflation to make, but not appropriate when you're trying to be pedantic.
Doesn't matter: there's no RGB model that captures the colour space. That exactly the reason CIE exists.
The early History of the Singular Value Decomposition (1993) [pdf]
21–30 of 85 posts
Re: The early History of the Singular Value Decomposition (1993) [pdf]
#22Earlier quoted context omitted.
ADAM is related if your second derivative matrix happens to be diagonal . Of course, it takes about 5 minutes to show that any DNN is going to have very very high magnitude off-diagonal terms by the way it's constructed, so pretending that a diagonal approximation is close enough is crazy.
Adam doesn't use the second derivatives matrix, it uses second moments of the gradient, which is the diagonal of the uncentered covariance matrix, but neither of them are directly related to SVD or singular values anyway. There is a slight connection where Adam approximates full-matrix Adagrad which computes inverse square root of the convariance matrix, which you usually do using eigendecomposition, but on the covar…
See the Fisher Information, and the Cramer-Rao Lower Bound (an inequality on how much the inverse covariance matrix and the Hessian can differ).
Re: The early History of the Singular Value Decomposition (1993) [pdf]
#23Re: The early History of the Singular Value Decomposition (1993) [pdf]
#24Earlier quoted context omitted.
> You are doing it wrong. I didn’t write any of it. I occasionally get assigned PRs written (or not, in this case) by other devs. > Claude can generate tests and verify the code better than humans now. It certainly didn’t do that in this case. > I don't trust humans to get things right anymore -- I have a PhD and Claude knows all the math and libraries better than me. If it knows all the libraries so well, why did it…
If you're getting code without tests to review in a PR, that should be an instant reject without even looking at the code .
Re: The early History of the Singular Value Decomposition (1993) [pdf]
#25Earlier quoted context omitted.
You seem to be conflating "RGB" with one particular RGB color space: sRGB. That's a common enough conflation to make, but not appropriate when you're trying to be pedantic.
Doesn't matter: there's no RGB model that captures the colour space. That exactly the reason CIE exists.
However the entire color space of CIE RGB (1931) includes points where some of the RGB components are negative.
Because positive components are sometimes desirable (e.g. because one can make light filters whose outputs are those components), the alternative XYZ representation is derived by computation from the original CIE RGB, which had been obtained from experiments with human subjects.
Any RGB model can capture the entire color space, even sRGB. The limitation is not at capturing, but only at reproducing colors when using RGB emitters, because the emitters cannot reproduce components with negative values.
There are no RGB models that can capture the entire color space without having points where some components are negative. This is caused mostly by defects of the human color vision, e.g. by the fact that the red receptors are also sensitive to violet light, not only to red light, and by the fact that the selectivity curves of the photoreceptors do not have ideal shapes.
Re: The early History of the Singular Value Decomposition (1993) [pdf]
#26Earlier quoted context omitted.
If you're getting code without tests to review in a PR, that should be an instant reject without even looking at the code .
Go ahead and have claude add and run units tests for you as part of the PR review process.
Re: The early History of the Singular Value Decomposition (1993) [pdf]
#27Earlier quoted context omitted.
Doesn't matter: there's no RGB model that captures the colour space. That exactly the reason CIE exists.
Since you seem to know, and I am curious, doesn't CIE[1] effectively use RGB to describe its space, too? Eg: the r̅(λ) g̅(λ) b̅(λ) color matching functions? Or is there something else in CIE you're referring to? [1] https://en.wikipedia.org/wiki/CIE_1931_color_space
Re: The early History of the Singular Value Decomposition (1993) [pdf]
#28Earlier quoted context omitted.
ADAM is related if your second derivative matrix happens to be diagonal . Of course, it takes about 5 minutes to show that any DNN is going to have very very high magnitude off-diagonal terms by the way it's constructed, so pretending that a diagonal approximation is close enough is crazy.
Adam doesn't use the second derivatives matrix, it uses second moments of the gradient, which is the diagonal of the uncentered covariance matrix, but neither of them are directly related to SVD or singular values anyway. There is a slight connection where Adam approximates full-matrix Adagrad which computes inverse square root of the convariance matrix, which you usually do using eigendecomposition, but on the covar…
Theorem 1, section 1.3, page 2 shows that the expected variance of the gradient of the loss function and the expected second derivative of the loss function are equal at the minimum. I hate that the ADAM paper did not talk about this, this is something that is hammered into anyone who has taken a mathematical statistics course. This has been an established fact in statistics for well over 100 years.
https://courses.grainger.illinois.edu/ece563/fa2025/Notes11-...
Away from the minimum they can diverge, but there is a close enough connection to make it an extremely useful approximation.
Re: The early History of the Singular Value Decomposition (1993) [pdf]
#29Earlier quoted context omitted.
> You are doing it wrong. I didn’t write any of it. I occasionally get assigned PRs written (or not, in this case) by other devs. > Claude can generate tests and verify the code better than humans now. It certainly didn’t do that in this case. > I don't trust humans to get things right anymore -- I have a PhD and Claude knows all the math and libraries better than me. If it knows all the libraries so well, why did it…
If you're getting code without tests to review in a PR, that should be an instant reject without even looking at the code .
Management didn’t like that the last time I was assigned a AI-gen PR and I like my job otherwise.
Re: The early History of the Singular Value Decomposition (1993) [pdf]
#30Earlier quoted context omitted.
If you're getting code without tests to review in a PR, that should be an instant reject without even looking at the code .
Go ahead and have claude add and run units tests for you as part of the PR review process.