Bounded Rationality (Posts about Hamiltonian)http://bjlkeng.github.io/enSun, 24 Apr 2022 01:03:07 GMTNikola (getnikola.com)http://blogs.law.harvard.edu/tech/rssHamiltonian Monte Carlohttp://bjlkeng.github.io/posts/hamiltonian-monte-carlo/Brian Keng<div><p>Here's a topic I thought that I would never get around to learning because it was "too hard".
When I first started learning about Bayesian methods, I knew enough that I
should learn a thing or two about MCMC since that's the backbone
of most Bayesian analysis; so I learned something about it
(see my <a class="reference external" href="http://bjlkeng.github.io/posts/markov-chain-monte-carlo-mcmc-and-the-metropolis-hastings-algorithm/">previous post</a>).
But I didn't dare attempt to learn about the infamous Hamiltonian Monte Carlo (HMC).
Even though it is among the standard algorithms used in Bayesian inference, it
always seemed too daunting because it required "advanced physics" to
understand. As usual, things only seem hard because you don't know them yet.
After having some time to digest MCMC methods, getting comfortable learning
more maths (see
<a class="reference external" href="http://bjlkeng.github.io/posts/tensors-tensors-tensors/">here</a>,
<a class="reference external" href="http://bjlkeng.github.io/posts/manifolds/">here</a>, and
<a class="reference external" href="http://bjlkeng.github.io/posts/hyperbolic-geometry-and-poincare-embeddings/">here</a>),
all of a sudden learning "advanced physics" didn't seem so tough (but there
sure was a lot of background needed)!</p>
<p>This post is the culmination of many different rabbit holes (many much deeper
than I needed to go) where I'm going to attempt to explain HMC in simple and
intuitive terms to a satisfactory degree (that's the tag line of this blog
after all). I'm going to begin by briefly motivating the topic by reviewing
MCMC and the Metropolis-Hastings algorithm then move on to explaining
Hamiltonian dynamics (i.e., the "advanced physics"), and finally discuss the HMC
algorithm along with some toy experiments I put together. Most of the material
is based on [1] and [2], which I've found to be great sources for their
respective areas.</p>
<p><a href="http://bjlkeng.github.io/posts/hamiltonian-monte-carlo/">Read moreā¦</a> (52 min remaining to read)</p></div>BayesianHamiltonianmathjaxMCMCMonte Carlohttp://bjlkeng.github.io/posts/hamiltonian-monte-carlo/Fri, 24 Dec 2021 00:07:05 GMT