Skip to content
 

Read this: it’s about importance sampling!

Importance sampling plays an odd role in statistical computing. It’s an old-fashioned idea and can behave just horribly if applied straight-up—but it keeps arising in different statistics problems.

Aki came up with Pareto-smoothed importance sampling (PSIS) for leave-one-out cross-validation.

We recently revised the PSIS article and Dan Simpson wrote a useful blog post about it the other day. I’m linking to Dan’s post again here because he gave it an obscure title so you might have missed it.

We’ve had a bunch of other ideas during the past few years involving importance sampling, including adaptive proposal distributions, wedge sampling, expectation propagation, and gradient-based marginal optimization, so I hope we can figure out some more things.

8 Comments

  1. Chris says:

    Man, what a low-effort blog this has become.

    “I hope we can figure out some more things”

    Just shut it down already if that’s what you have to offer.

    • Andrew says:

      Chris:

      Just in case you’re not kidding with this comment, let me explain.

      Sometimes we come up with ideas which it takes us years to work out. Some of these ideas end up being directly useful, others can become useful indirectly by motivating new ideas. For example, the method described in our article, Adaptively scaling the Metropolis algorithm using expected squared jumped distance, did not seem to have much practical value, but the ideas behind it were an inspiration for the Nuts algorithm, which has been super-useful. Often we’re in the stage of having some partial ideas and hoping we can figure out more.

      One of the purposes of this blog is to share research progress, as with Pareto-smoothed importance sampling linked above. Another purpose is to give others a sense of what research feels like. Part of this is that research is uncertain. “I hope we can figure out some more things . . .”: That might be disappointing for you to hear, but that’s how it goes sometimes. The ideas don’t all come at once.

      Also I recommend you think a bit about variation between people. You are one reader; other readers have other interests. Some people have told me they can’t stand the posts about politics because they hate politics. Other readers have no interest in sports, but I keep posting on sports, etc. If you dislike posts that include hopeful statements about future research, that’s fine: you’re just one more reader with personal preferences. I’m not planning to “shut it down already” but you don’t need to read the posts you don’t enjoy.

      • Martha (Smith) says:

        “Part of this is that research is uncertain.”

        The emphasis on uncertainty is perhaps the most important thing this blog gives that attracted me to it originally and that keeps me coming back.

    • Matti Nussi says:

      Bah. I like this blog since sometimes there are these “simpler” posts. I hope Andrew keeps this up!

    • Chris.

      I believe the ideas behind importance sampling are essential for understanding statistics but they are unfortunately often overlooked.

      This has come up a number of times on this blog e.g. https://statmodeling.stat.columbia.edu/2009/05/27/the_benefits_of/#comment-48904

  2. Kyle C says:

    Andrew, please do not shut the blog down. I would be willing to pay double the current subscription price. I am sure I am not alone.

  3. andrés says:

    These are my favorite posts 🙂

Leave a Reply