Viewing profile — dkislyuk

I agree that "it has nice derivatives" is a great empirical reason to use a specific function in ML, but it doesn't sufficiently prove that it's the best function to use. And even …

(meant to say, scale-invariance of probability ratios, or shift-invariance of the inputs)

Something that really helped me grasp the foundational relevance of the softmax is to justify from first principles why e^x shows up in the preferred mapping function in the numera…

Softmax is defined over an arbitrary vector of raw real numbers. Stating that those inputs are "logits" is applying post-hoc semantics to what the model is learning. One of the key…

The Paul Cooper production is great. The Rest Is History also just finished a long series (spread out in three seasons, starting on episode 421) on the Punic wars, similarly well d…

Presenting information theory as a series of independent equations like this does a disservice to the learning process. Cross-entropy and KL-divergence are directly derived from in…

From Walter Isaacson's _Steve Jobs_: > One of Bill Atkinson’s amazing feats (which we are so accustomed to nowadays that we rarely marvel at it) was to allow the windows on a scree…

Pinterest | Hybrid @ {San Francisco, New York, or Seattle} | Full-time + internships Pinterest’s Advanced Technologies Group (ATG) is an ML applied research organization within the…

This is a great characterization of self-information. I would add that the `log` term doesn't just conveniently appear to satisfy the additivity axiom, but instead is the exact his…

I think commodification is directly tied to a perceived drop in quality. For example, if the barriers to making a video game keep going down, there will be far more attempts, and p…

Presumably the book from this thread by Charles Petzold will be a great canonical resource, but originally there was a quote by Howard Eves that I came across that got me curious: …

Yes, but such a property was not available to Napier, and from a teaching perspective, it requires understanding exponentials and their characterizations first. Starting from the o…

I found that looking at the original motivation of logarithms has been more elucidating than the way the topic is presented in grade-school. Thinking through the functional form th…

Pinterest | San Francisco, New York, or hybrid/remote (US-only) | ML Engineer / Applied Research Scientist | Full-time Pinterest’s Advanced Technologies Group (ATG) is hiring for a…

Pinterest Advanced Technologies Group | Staff Engineer, iOS and applied ML | US remote or hybrid in SF/NY | Full-time We’re looking for strong engineers to help us build consumer A…

Yes, exactly. ViTs need O(100M)-O(1B) images to overcome the lack of spatial priors. In that regime and beyond, they begin to generalize better than ConvNets. Unfortunately, ImageN…

Rocket Men by Robert Kurson tells the Apollo 8 story in a captivating manner. Some of the passages are quite dramatic but it's justified given the litany of firsts accomplished by …

In the current world, deep learning with homogeneous computation graphs, tuned with backprop, has won the Hardware Lottery [1]. This is unfortunate for research outside of that are…

Moravec's paradox is the usual counterargument given to this line of reasoning. We've had far less progress in embodied robotics, where a robot has to interact with the real world …

Uber sold off its own self-driving division back in 2020: https://arstechnica.com/cars/2020/12/uber-sells-self-driving... so this kind of partnership has probably been looming for …

The one distinction I would add with neural networks is that it's not just a recursive tree traversal that one would get when evaluating an arithmetic statement, but an actual grap…

As another commenter said, viewing a neural network as a computation graph is how all automatic differentiation engines work (particularly reverse-mode where one needs to traverse …

The LLM / foundation model industry will happily stay in the United States and ignore the stifling regulation of the EU if necessary. There is so much market share to capture domes…

> These platforms are for casuals. At least in road and trail running, this doesn't line up with my observation. There are plenty of elite athletes, at least in the US, who are pos…