Viewing profile — jing

No it can’t. That’s d to d

In practice the functions just need to be piecewise differentiable. The RELU is the canonical example for deep learning. At kinks a subderivative is used.

Why not just get the 7702p?

Actually I'm pretty sure the Rome numbers are for double precision whereas most numbers quoted for GPUs are for single precision or less, making Rome's 3.4tf even more impressive.

https://www.fast.ai/2018/07/02/adam-weight-decay/

Particularly since it would not be unreasonable to assume that the "mi" in mimalloc is incorrectly pronounced like the "mi" in Microsoft.

Yep. And Mellanox does it too fwiw - https://community.mellanox.com/s/article/vma-improves-redis-...

I've got the 9550 and run Ubuntu on it. No issues whatsoever and I do very compute-intensive work on it. One thing that I've found is very important is to clean out the fans often,…

I think this resource could be helpful: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ Essentially, RNNs and feed forward networks are very similar - RNNs are just "unro…

Note that it's not just the loss function. It's the loss function combined with a very specific problem formulation - namely a neural network with only linear activations (equivale…

I find it hard to believe that SGD would be faster than the closed form solutions for linear regression (gels, gelsd etc.). The closed-form solutions give a lot of other benefits i…

The Xeons support much more RAM.

Are physics engines not yet accurate enough to enable "virtual" pre-training / full training of the networks, lighting conditions, etc? If they are, exclusively using physical robo…

DAC is an order of magnitude cheaper than optical and might be fine for their use based on their comments of cat6 v cat6a and "short runs in their lab". Also, if they are using two…

Some claim that this is part of a broader trend of "generational theft" from the young to the old and have some interesting slides to demonstrate their thesis: http://online.wsj.co…

The pairing idea isn't new although it does seem to be very successful - there's a company called Thinkful ( http://www.thinkful.com ) that does online learning by pairing a mentor…

ZFS may have been, but not the 128 bit support. I mentioned that there are "admittedly much more relevant features" which are in fact what smugmug likes: "ZFS is the most amazing f…

Yes, this is theoretically cool and all, but when I hear something like: "Let's start with the easy one: how do we know it's necessary? Some customers already have datasets on the …

A recent HN post which may be helpful for you: http://news.ycombinator.com/item?id=1752139

Here's a few I've really liked that have a broader subject matter than only what you asked for. I figure that if you like to read you'd like these books. --- NON-FICTION --- Darkne…

I'm curious - you say, "A do-it-yourself wedding invitation kit costs $45, while professional wedding invitations are hundreds or thousands of dollars." How much of that "hundreds …

If it was profitable to harvest / clean up the spill, wouldn't BP be doing it already? For that matter, wouldn't all of its competitors such as Exxon be doing the same as well?

Exactly. Unfortunately, since Palantir is in the enterprise space, few people on HN immediately know of the how 1) cool and 2) difficult the problems and solutions in the space are…

I don't mean to sound rude, but your comment shows how little you know understand about securities, investing, risk, and reward. Of course nobody comes in asking for "crappy mortga…

Does that mean that we should throw the CEO of McDonald's in jail too because he knowingly sells junk food to customers? No, we shouldn't. And the reason is that people are free to…