Viewing profile — mjw

> A minor key has sharps and flats to create tension. The natural minor or Aeolian mode doesn't use any notes outside the diatonic scale (probably what you meant by "sharps and fla…

This is very neat. That said the reason these methods haven't received much attention so far is that relatively few people actually need to compute Jacobeans or Hessians directly. …

When I started out in ML I was really keen to learn about the most 'mathsy' approaches out there. I think with hindsight, it's great to have a broad spectrum of methods available t…

Ah yep, true. I'd forgotten you can still get the saddle effect from higher-order derivatives, the Hessian eigenvalues aren't enough to characterise it. I was thinking of examples …

My main quibble from this paper is: > For deeper networks, Corollary 2.4 states that there exist “bad” saddle points in the sense that the Hessian at the point has no negative eige…

See the other replies above, but: the logistic has heavier tails than the normal, so might do better in cases where we need robustness, where unexpected outcomes remain possible ev…

Ah yep, I forgot it's the canonical link. That's more of a small computational convenience though, right, at least when fitting a straightforward GLM -- it should be very cheap to …

Their answer is pretty much 'because it's based on the log-odds', which to me is still only very mild motivation. There are other non-linearities which people use to map onto (0, 1…

Pretty much any kind of mathematical modelling that involves uncertainty, really. Making inferences and predictions from data, in the presence of uncertainty. Analysis of the prope…

If anything, to me a lot of deep learning literature seems to lack the statistical insight and theory that's available to other subfields in machine learning (whether the Bayesian/…

It's more an empirically-verified thing than a mathematical fact, there's nothing magic about 16 bits AFAIK. Empirically 16 bits seems to work well enough for some tasks, taking it…

Yep. To elaborate: really big batch sizes can speed up training data throughput, but usually mean that less is learned from each example seen, so time-to-convergence might not nece…

Warp-CTC implements one specific model (or at least, one specific loss function), it's not really a general framework in the same way as the other libraries mentioned.

> If I publish a paper people want to know what the data I found suggests and that's it. What they're going to get, is what your data and your modelling assumptions suggest. If you…

It's odd to complain specifically about subjectivity of the prior when the likelihood is often just as subjective. Gelman puts this well here: http://andrewgelman.com/2015/01/27/pe…

On the whole this is useful, although I think it's a little unfair to Theano in places. * Performance I feel they should score separately here for compilation/startup time vs runti…

Agreed that SQL is ugly as hell, but if you want to talk about its theoretical properties that's a separate debate. Theory doesn't care whether something's aesthetically pleasing, …

Does anyone know if TensorFlow can apply algebraic simplifications and numerical optimisations to the compute graph, in the way that Theano does with its optimisations? Sounds like…

Ah OK, fair enough. Not sure what definition you're using for the category of tables, but I don't think the distinction between table and query is really that significant, at least…

> What I mostly see around is just standard linear algebra operations on matrices and vectors lifted to higher-dimensional tensors point-wise Equally what is matrix multiplication …

Eh... SELECT a.*, b.* FROM a CROSS JOIN b; (SELECT 'a' AS tag, * FROM a) UNION (SELECT 'b' AS tag, * FROM b);

AVDI is incredibly awesome. Thanks so much for this work. Really nice readable paper on it too. As someone working with large datasets I think automating variational inference + SG…

It's nice to get people thinking about possible connections and sharing terminology here, but I'm not sure that many of the connections which the article manages to make precise ar…

When you penalise the L2 norm of the convolution of the image with a filter (like a gradient or edge detector for example) you are effectively doing this. The spectrum of the filte…

I've heard this too and it's a great way to demonstrate you don't really know what statistics is :) Statistics is not (just) opinion polling, there's a lot more to it than estimati…