My Favorite Algorithm: Metropolis-Hastings • Hacker News Board

Re: My Favorite Algorithm: Metropolis-Hastings

#21

Post by murbard2 » Tue, Jun 02, 2015, 2:35 AM UTC

stellographer wrote: post #20

murbard2 wrote: post #12

Earlier quoted context omitted.

No! The point of Metropolis-Hastings is to sample from a distribution when you do not know the partition function. It is the most important building blocks in a set of algorithms broadly known as Markov Chain Monte Carlo. These algorithms are particularly useful when performing Bayesian statistics. Genetic algorithms will not give you samples from a distribution, they only perform optimization. Particle swarms also f…

I see you feel strongly about this. "The point of Metropolis-Hastings is to sample from a distribution when you do not know the partition function." That's one point, yes. The other is optimization. In which case I prefer the others I mentioned. "they do not seem to have either theoretical justification or empirical success." That's just patently false. They _do_ have empirical success. And besides, a lot of these va…

1) Can you point out where particle swarm optimization has been successfully applied? Papers specifically about particle swarm optimization carry very little weight as it is very easy to design toy problems where any optimization technique is going to perform well, I'm looking for actual, practical use.

2) When you are sampling a distribution, you're not trying to make a 6-decker grilled cheese, you're trying to make many grilled cheese sandwiches.

3) In completely new ways? Not really. Algorithmic complexity which dominates run time independent of the computing medium. An algorithm designed to be efficiently run by a group of human "computers" with calculators is probably very similar to the same algorithm designed to be run by a CPU. If anything, the CPU optimized algorithm are likely to benefit from more sequential processing and less parallelism.

Re: My Favorite Algorithm: Metropolis-Hastings

#22

Post by eru » Tue, Jun 02, 2015, 6:12 AM UTC

gfodor wrote: post #15

I always go back and forth on which of the two things are more amazing: the fact that we can accurately estimate the rendering equation in tractable time, or the fact that the universe manages to do it in real time.

The universe can take all the time it wants. You wouldn't notice.

Re: My Favorite Algorithm: Metropolis-Hastings

#23

Post by stellographer » Wed, Jun 03, 2015, 1:15 AM UTC

murbard2 wrote: post #21

stellographer wrote: post #20

Earlier quoted context omitted.

I see you feel strongly about this. "The point of Metropolis-Hastings is to sample from a distribution when you do not know the partition function." That's one point, yes. The other is optimization. In which case I prefer the others I mentioned. "they do not seem to have either theoretical justification or empirical success." That's just patently false. They _do_ have empirical success. And besides, a lot of these va…

1) Can you point out where particle swarm optimization has been successfully applied? Papers specifically about particle swarm optimization carry very little weight as it is very easy to design toy problems where any optimization technique is going to perform well, I'm looking for actual, practical use. 2) When you are sampling a distribution, you're not trying to make a 6-decker grilled cheese, you're trying to make…

just downvoting every one of my comments because you disagree with them? That's cool.

1) Just one example? Here: http://adslabs.org/adsabs/abs/2009PhDT.........4C/

2) You are trying to find the global maximum. How do you not understand the value of communication when searching for a maxima in the likelihood distribution? You're just being intentionally obtuse.

3) Yes, really.

A story:

You have a landscape with mountains and hills and you have one person trying to find the tallest mountain. That person is blind, they can't see shit. That person is also mute and deaf. Their only sense is a vibrating altimeter. They get drunk, and climb mountains for 100000 days, trying to find the tallest mountain.

You are advocating the idea that you should send 1000 of these blind deaf mutes out there one at a time (running these in parallel is just faster serial) and then they should vote on which mountain is the tallest at the end.

I'm saying you should send a bunch of not-deaf-mutes out there (ie implement mutation, breeding, cross-contamination, gravitation, whatever) so they can tell each other where the stupid mountains are (this requires _parallel_) and they don't waste their whole time stumbling around (burning in).

You're just being intentionally argumentative.

Re: My Favorite Algorithm: Metropolis-Hastings

#24

Post by murbard2 » Wed, Jun 03, 2015, 1:59 PM UTC

stellographer wrote: post #23

murbard2 wrote: post #21

Earlier quoted context omitted.

1) Can you point out where particle swarm optimization has been successfully applied? Papers specifically about particle swarm optimization carry very little weight as it is very easy to design toy problems where any optimization technique is going to perform well, I'm looking for actual, practical use. 2) When you are sampling a distribution, you're not trying to make a 6-decker grilled cheese, you're trying to make…

just downvoting every one of my comments because you disagree with them? That's cool. 1) Just one example? Here: http://adslabs.org/adsabs/abs/2009PhDT.........4C/ 2) You are trying to find the global maximum. How do you not understand the value of communication when searching for a maxima in the likelihood distribution? You're just being intentionally obtuse. 3) Yes, really. A story: You have a landscape with mounta…

HN does not allow downvoting replies to one's comment, so someone else must be downvoting you. Your tone is aggressive and you display a poor grasp of the topic which is probably why you're being downvoted.

1) This is an abstract of a PhD thesis which makes no mention of particle swarms, I couldn't find the full text. This is your evidence?

2) No this isn't about looking for the global maximum. Several people have explained to you that this is a sampling algorithm, but you still fail to show understanding of the difference.

3) Your story doesn't demonstrate your point and you do not understand the argument I presented to you. The types of algorithms suited for an army of people with calculators 100 years ago isn't fundamentally different from the type of algorithm suited for computers today, and if anything, it's likely to be more sequential, not less.