Viewing profile — apl

Neural architecture search (NAS) is a thing! But it's almost exclusively based on meta-gradients. Again, wouldn't put my money on GAs ever outperforming gradient-based methods agai…

An AlexNet/ResNet-type moment may be in the cards for GAs, but I wouldn't put any money on it. They're typically only one better than brute-force. This can be good enough (and is c…

Several hints here are severely outdated. For instance, never train a model in end-to-end FP16. Use mixed precision, either via native TF/PyTorch or as a freebie when using TF32 on…

Hard disagree. V100s are a perfectly valid comparison point. They're usually what's available at scale (on AWS, in private clusters, etc.) because nobody's rolled out enough A100s …

You can almost 1:1 translate this by swapping "tf" and "torch". No need to use nn.Conv2d -- there's a functional API for all these layers: https://pytorch.org/docs/master/nn.functi…

> gradient descent no longer has to be written by hand Nobody's been writing derivatives by hand for 5+ years. All major frameworks (PyTorch, Tensorflow, MXNet, autodiff, Chainer, …

Mainly because it is genuinely exhausting for any medical practitioner. That lots of patients "enjoy" googling symptoms and coming up with far-fetched self-diagnoses is a given. Bu…

For this particular problem, Mask R-CNN would have been the way to go -- it spits out instances as opposed to just deciding, for each pixel, to which class it belongs. Or an SSD (i…

There's many perspectives on everything. Deep ConvNets, for instance, can be expressed as a continuously evolving ODE. Here's a fantastic paper on this view: https://papers.nips.cc…

Yeah, that's right. Doing too little proofreading with HN comments...

Their applications are tabular data (for which MLPs have never been the method of choice) and MNIST (which I could classify at 85% with a rusty nail), so it's not super impressive.…

This represents the absolute worst kind of science journalism, completely devoid of context and domain knowledge. Virtually every definitive statement in here is wrong. Their expla…

Running Windows is perfectly fine; the major libraries for GPU-accelerated autodiff and networks (CUDNN with Pytorch or Tensorflow) have great support nowadays. It's the AMD GPU th…

That's not quite how the process works. These papers go through multiple (> 2) revisions. At any iteration, there's ample opportunity for updating references. This applies double g…

It's certainly an interesting paper, but there's a bit of publication weirdness at play here. In October '17, Cueva & Wei put out a(n anonymous) paper that recapitulates the core r…

> I've noticed that when I get a blink/flinch response from > something (usually some sand or a bug hitting my face when > I'm on the bike), it feels like I blink just a split seco…

As someone who works in visual neuroscience, this article's a tough read. Lots of statements that are semi-accurate at best. 1) Eyes don't work like cameras; there's no real "expos…

Thanks for coding this up. The code in the blog post is, at best, incredibly misleading.

I only skimmed the notebook, but the code looks fairly inadequate. The pandas portion, for instance, treats the data frame as a dumb array and critically ignores grouping functiona…

We didn't have to reverse-engineer the CPU. Also, CPUs are rationally designed; we have zero guarantees that an evolved brain follows any principles at all. Every brain region may …

This came out two days ago and uses what they call intelligent synapses to improve multi-task learning: https://arxiv.org/abs/1703.04200 Seems closely related.

Try this: https://www.ncbi.nlm.nih.gov/pubmed/26234212 I guess the key lesson is -- don't rely on a single approach, because its limitations may well lead you astray. Applies to co…

Even in these apparently simple feedforward sensory networks, connectomics haven't been the anticipated panacea. There's been a flurry of follow-up papers to Takemura et al., essen…

Cortical layers are in no way equivalent to DNN layers.