Viewing profile — duvenaud

This is simply wrong. Backprop has the same asymptotic time complexity as forward.

Which material ongoing issues are we ignoring? The paper is mainly talking about how the mundane problems we're already starting to have could lead to an irrecoverable catastrophe,…

One of the authors here. I don't think we anthropomorphize AI as some sort of God. Here's a more prosaic analogy that might be helpful. Imagine tomorrow there's a new country full …

> as we notice these things, we pass laws against it Well, the claim is that that's the sort of thing that will get harder once humans aren't involved in most important decisions. …

> we have categorized the ways it goes rogue (monopoly, extortion, etc) and responded adequately. This objection is a reasonable one. But the point of the paper is that a lot of th…

> A society whose tagline is "alive and free in some senses" is already dystopian! Haha. Well we might agree about that - that description covers a wide range of possibilities. If …

I appreciate your engagement, and I don't really have a plan myself to address these problems, but I don't really know what to do with "Socialism" as a recommendation. Care to elab…

These are all good points about our use of language, thanks for the feedback. Maybe "disempowerment" is a bit of a red herring, or a misleading problem to focus on. The reason we d…

Yep, a major missing piece in this entire problem / discussion is how to characterize how much "power" "humans" have had. My best idea so far is to characterize the sorts of outcom…

> What incentives do any humans have to so totally delegate the functioning of the core levers of societal power that they're unable to prevent their own extinction? Because it'll …

Last author here. I agree that states already have little incentive to effectively represent their citizens. But they could have even less! What would it look like to face these is…

In many senses, yes. But the empowered ones still needed to keep most of the rest of the people happy and healthy enough to work, most of the time. That's what we're saying will ch…

Last author here. Good point, I agree that the move to an entirely self-sustaining machine economy would require extra time, and that would drag out the time to extinction even the…

Update: the code is here: https://github.com/jerryqhyu/distill_bayes_net

I agree that priors over aspects of the world would be more useful, but I don't think that they're important in making natural intelligence powerful. In my experience, the importan…

BNNs certainly have their uses, but I think people in general found that it's a better use of compute to fit a larger model on more data than to try to squeeze more juice from a gi…

Thanks for pointing that out!

This approach characterizes a different type of uncertainty than BNNs do, and the approaches can be combined. The BNN tracks uncertainty about parameters in the NN, and mixture den…

I still am excited by Dex ( https://github.com/google-research/dex-lang/ ) and still write code in it! I have a bunch of demos and fixes written, and am just waiting for Dougal to …

I think we used a distill.pub template. Also Jerry wrote some custom BNN fitting code in javascript. I'll ask my co-authors to open-source it.

Sure, instead of saying "choose" a prior, you could say "elicit". But I think in this context, focusing on a practitioner's prior knowledge is missing the point. For the sorts of p…

I would argue against the idea that "MLE is just Bayes with a flat prior". The power of Bayes usually comes mainly from keeping around all the hypothesis that are compatible with t…

Good point. We wrote this pre-double descent, and a massively overparameterized model would make a nice addition to the tutorial as a baseline. However, if you want a rich predicti…

I agree that Bayesian neural networks haven't been worth it in practice for many applications, but I think the main problem is that it's usually better to spend your compute traini…

I agree choosing priors is hard, but choosing ReLU versus LeakyReLU versus sigmoid seems like a problem with using neural nets in general, not Bayesian neural nets in particular. A…