Believe your models (up to the point that you abandon them)

In a discussion of his variant of the write-a-thousand-words-a-day strategy (as he puts it, “a system for the production of academic results in writing”), Thomas Basbøll writes:

Believe the claims you are making. That is, confine yourself to making claims you believe. I always emphasize this when I [Basbøll] define knowledge as “justified, true belief”. . . . I think if there is one sure way to undermine your sense of your own genius it is to begin to say things you know to be publishable without being sure they are true. Or even things you know to be “true” but don’t understand well enough to believe.

He points out that this is not so easy:

In times when there are strong orthodoxies it can sometimes be difficult to know what to believe. Or, rather, it is all too easy to know what to believe (what the “right belief” is). It is therefore difficult to stick to statements of one’s own belief. I sometimes worry that our universities, which are systems of formal education and formalized research environments, have become too orthodox. I’m sure the orthodoxies are largely true and justified. I’m just not sure we’re giving each other, and ourselves, the time we need to believe in them.

Basbøll’s comments about belieiving what one writes (which, I think might be even better labeled as “provisional belief”) reminds me of some things I’ve written with Shalizi (for example, this) on the importance of strong assumptions in statistical learning. In short, strong assuptions play two roles: First, with strong assumps we can (often) make strong and precise inferences. The likelihood function is a powerful thing. Second, strong assumptions are strongly checkable and falsifiable. We take our models seriously, work with them as if we believe them unquestioningly, then use the leverage from this simulation of belief to check model fit and explore discrepancies between inferences and data.

This is only vaguely related to Basbøll’s idea, but I think there is some connection in that it illustrates the two-edged nature of belief. On the one hand, belief is powerful. By conditioning on assumptions, we can rule out alternatives and move quickly and surely. But belief is risky, especially since all of our beliefs, if stated precisely enough, our false. The resolution is that we can use the strength and power of beliefs to better study their limitations.

P.S. I think Basbøll’s argument would be much stronger if he could support it with a factual story involving mountaineering. In the absence of such a story, he has to present his claims as raw speculation backed only by his introspection and personal experience. I have to admit, though, that it’s hard to come up with such stories. Perhaps he find something written by a poet who’d framed the story in just the right way. But if Basbøll were to use someone else’s writing, he’d have a dilemma: cite the source and then reveal it’s just a story, not something that actually happened; or don’t cite any source and then maybe get busted for copying someone else’s work without attribution. In the modern era of Google, that’s not so easy. This could be a serious blow to the sort of sensemaking scholarship that is Basbøll’s specialty.

P.P.S. This cut-and-paste is just killing me. If I do any more posts on this guy, I’m gonna have to figure out how to type ø directly!

16 thoughts on “Believe your models (up to the point that you abandon them)

  1. P.P.S. This cut-and-paste is just killing me. If I do any more posts on this guy, I’m gonna have to figure out how to type ø directly!

    I don’t know what system you work with, but on the mac it’s just option-o. The Internet seems to agree that on Windows you can type Alt-0248. I have no idea for linux.

  2. Alt+0248 should yield “ø” on a Windows machine. Alternatively, you could use [ö], as they’re not phonetically distinct in Danish – not sure what Mr Basbøll would think about that, though.

    I think statistical inference is particularly vulnerable to the Gettier problem of whether justified true belief is really knowledge (see Gettier 1963, “Is Justified True Belief Knowledge?”, Analysis 23). Alvin Goldman’s demand of a causal connection solves some of the epistemological issues, but seems difficult to implement in practical research (Goldman 1967, “A Causal Theory of Knowing”, The Journal of Philosophy 64). After all, we knew that tobacco smoking was carcinogenic decades before we had an idea exactly how.

  3. Analytic philosophers and their contemporary holdovers are responsible for the idea of defining:
    Subject S knows proposition P as:
    (1) S knows that P: iff
    P is true
    S believes P.

    Note: the goal (the whole goal) is definition, analytic—based on language, intuition, and counterexamples.
    Finding zillions of counterexamples, they moved on to adding various requirements to (1) like:
    (2) P was obtained through a reliable or justifiable method. But that too is open to problems, e.g., the Gettier example that a commentator mentioned:

    The poster says P: Gelman will talk at Handsworth U on April 20, 2012.
    Typically posters are reliable. That talk was cancelled, but it happens (long roundabout story) that Gelman is speaking at Handsworth U on April 20, 2012, so P is true, even though our reliable procedure failed in this case. Intuitively, do we want to say that subject S, who reached her belief based on the poster, knows that Gelman is speaking at Handsworth U on April 20, 2012?
    The intuition is no, she didn’t really know it, even though it satisfies the definition. So thousands of papers have been written to add yet more requirements to the “S knows that P” definition, until the whole thing was sort of abandoned, but only by advancing similar attempts (I am not entirely up on the latest).
    No one says what a reliable method is, whereas statisticians do. So I hope statisticians won’t take their epistemological lessons from the blind alleys of analytic philosophers. (And I hope analytic epistemologists will look to statistics to try to implement their definitions.)

    But now we get to what I would regard as the real moral of the person Gelman is quoting (as to why universities tend to be orthodox, and people often will not abandon a paradigm even if it’s run out of steam, and they know it but can publish in it). I’m being very quick here, and will likely get in trouble with philosophers.

  4. In Linux, you set up your compose key by including a line in your xorg.conf file like this:
    Option “XkbOptions” “compose:ralt”
    Then the right alt key becomes the compose key. To enter an ø, press the compose key, then o then /. You can enter all kinds of other useful characters in the same way: ¢ is compose, c, /, € is compose, =, e. There’s a wikipedia page for Compose_key that has the whole list.

  5. Mayo’s comment reminded me of a conversation I had many years ago on the analytic epistemology blog, Certain Doubts. In response to post by Michael Huemer about the “knowledge norm” for belief, Gregory Wheeler introduced the belief and/or knowldege of Ichiro Suzuki about whether or not he will git a given ball. The question is, in part, whether his batting average can justify his belief. The discussion begins here here.

    • Thomas, not sure I see the connection, although I couldn’t read the entire Wheeler exchange. I didn’t realize people were blogging in 2005–early innovators. Your writing instruction looks intriguing.

      • Yeah, we were doing it before it was cool! The connection is very superficial: something about the justification of beliefs and statistics and analytic philosophy. Rereading it, I have to admit I can’t even really follow my own argument in that discussion. But my response to Gettier-style objections to JTB is that P means different things in the two cases. What the poster meant when he wrote (falsely) “Gelman will talk at Handsworth U on April 20, 2012,” is not what we mean when we say (truly) “Gelman will talk at Handsworth U on April 20, 2012.”

  6. Andrew: The stuff following after “On the one hand” does resemble Peirce’s “123 waltzing”

    The 1 being make strong assumptions, 2 test those assumptions against brute force reality and 3 being getting yet a new set of assumptions in light of 1 and 2. The Peircean concept of doing science once being being likened to a dizzying waltz of 123, 123, 123, …. (indefinitely).

    Thomas: Ian Hacking wrote about Peirce’s defence of using probability for believe if a given situation through the playing of a card game with the devil about one soul (that cannot be repeated) and called it the faith, hope and charity argument (which also was used to justify the indefinitely above.)

  7. Pingback: “Any old map will do” meets “God is in every leaf of every tree” « Statistical Modeling, Causal Inference, and Social Science

  8. Pingback: Colorless green facts asserted resolutely « Statistical Modeling, Causal Inference, and Social Science

Comments are closed.