Viewing profile — _hark

Yeah. The safety of the message is underwritten by its state sanction.

You literally can do a kind of model PCA, using the Hessian (matrix of second derivatives of the loss function w/r/t the parameters, aka the local curvature of the loss landscape),…

I don't recall Andrej making "next year!" claims, it was always Elon. I found Andrej's talks from that time to be circumspect and precise in describing their ideas and approach, an…

They really should have just marketed the software "as-is" to whatever extent that is allowed by law. I guess they didn't because deployed automobile software is probably not allow…

https://archive.ph/GWBEl

There aren't merit-based scholarships to any Ivy League schools, they all offer need-based financial aid packages.

I'm a researcher at Oxford, and I've both taught and studied here and in the US. The undergraduate teaching here is phenomenal. It's incredibly labor intensive for the staff, but t…

I sat the All Souls exam, taking the philosophy specialist papers, though I'm a math/physics/ML guy. It was a lot of fun, I really appreciate that there's somewhere in the world wh…

Very cool! I've done research on reinforcement/imitation learning in world models. A great intro to these ideas is here: https://worldmodels.github.io/ I'm most excited for when th…

If FPGAs are competitive on perf/watt, why aren't they more widespread (other than crap software tooling)? Honestly I've asked different hardware researchers this question and they…

Can anyone comment on where efficiency gains come from these days at the arch level? I.e. not process-node improvements. Are there a few big things, many small things...? I'm curio…

Entropy is not absolute! The entropy of some data is well-defined with respect to a model, but the model choice is free. I.e. different models will assign different entropy to the …

Maybe a correction is needed. Academia has become so gamified. It's supposed to be about ideas, truth, beauty. Too many are in it for the prestige, which has ironically made it les…

Interesting. You need some local structure with global coherence. But you want it to be complex, not too regular. Like a Penrose Tiling.

Hmm. At least it's still fine to define limits of the complexity for infinite strings. That should be unique, e.g.: lim n->\infty K(X|n)/n Possible solutions that come tom mind: 1)…

Interesting. I guess then we would only be interested in the normalized complexity of infinite strings, e.g. lim n-> \infty K(X|n)/n where X is an infinite set of numbers (e.g. the…

This could also be a hardware startup. If only there were some entrepreneur types around... Presumably there's a market for this in other niches, e.g. weather monitoring, defense/b…

The issue with the invariance theorem you point out always bugged me. Let s be an algorithmically random string relative to UTM A. Is it the case that there exists some pathologica…

I agree with you in spirit. I just thought you might be interested in some of the technical details regarding the relationship between compression and generalization! I'll have a p…

Great point. This points to the related issue: what do we want to compress? Do we want to compress "the answer", here the arithmetic expression's solution, or do we want to compres…

Say we have some dataset composed of D bytes. Next, say I find some predictive model of the data M, where M is composed of N bytes. Furthermore, let us say that the entropy of the …