41–50 of 95 posts
Earlier quoted context omitted.
You can also say backpropagation is the chain rule from centuries ago.
It is a computationally clever application of the chain rule to minimize the amount of computation needed to compute gradients for all parameters in the network.
IMO backprop is the most trivial implementation of differentiation in neural networks. Do you know an easier way to compute gradients with larger overhead? If so, please share it.
Earlier quoted context omitted.
S-expressions are basically an AST with minimal parsing. It’s very convenient to work with.
Yeah, that's basically my objection. The convenience seems to hide something important. I know you can write any program with LISP, but using LISP for this kind of thing seems to be a symptom of limited applicability.
E.g. https://hylang.org/
Earlier quoted context omitted.
Yeah, that's basically my objection. The convenience seems to hide something important. I know you can write any program with LISP, but using LISP for this kind of thing seems to be a symptom of limited applicability.
I don't share this concern. You can easily write Python, Typescript and C++ in S-expression format since you're basically writing an AST, that's the whole point. Conversion between one and the other is... maybe not trivial, but certainly not rocket science. E.g. https://hylang.org/
I think another big thing that makes Lisp-like languages (Common Lisp, Clojure, Racket, ...) convenient for this kind of work is the fact that everything is an expression. Having "statements" and "expressions" separate makes writing some programs awkward. I use Clojure at my day job, and the way I write code takes for granted that I can simply bind `foo` to the result of a `case` or an `if` without ad-hoc operators that have completely different syntax from their statement-equivalents.
Earlier quoted context omitted.
Or, raster and vector images are philosophically the same thing . The only difference is that vector has more operations than raster. Raster just has "draw unit square at integer coordinates".
On the other hand, A Pixel Is Not A Little Square[0] would disagree, a raster is a grid sample of a continuous function. [0] http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf
Pictures of cats, subpixel rendered text, or company logo SVGs as displayed on a web page are point samples and not little squares.
User interfaces are good examples of often being little squares. Calling HN's beige background a point sampled discrete representation of an underlying continuous function seems pretty tortured — to me it seems like a bunch of beige little squares.
Surprised to see Stuart Russells name on this as I thought he was fully consumed by the doomsday cult. Although he's last author so he's probably only on it because he's the head of the lab
A lot of doomers work on AI. While frowning, and shaking their heads very gravely, so you know they don't approve.
Earlier quoted context omitted.
That would be mind blowing. Why go through all the lost intermediary steps, especially through Python and JS, when you can generate machine code directly
If your model is error-prone, having control structures, types and other compile-time checks is very valuable. It's harder to constrain arbitrary machine code to make something sensible.
It's funny, this kind of subtree mutation was looked at pretty deeply by Koza and Adamı in the 90s under the rubric of Genetic Algorithms, but with a slightly different optimization function One ref in the paper to 2000 for GAs for fast generation of program trees, but that's missing the main show Hope they're reading this and dig into those guys work
Surprised to see Stuart Russells name on this as I thought he was fully consumed by the doomsday cult. Although he's last author so he's probably only on it because he's the head of the lab
I've read several times that the author listed last is usually the most significant contributor, and the first author the least significant, due to some kind of tradition around modesty plus favourably introducing new names. (Which then of course doesn't work, if everyone knows it's happening...)
Here, you've interpreted it as the reverse, and by that I mean in the sensible way - did you not know about the tradition, or are you wrong? And how can we know either way for sure?