Tell me what you don’t know

We’ll ask an expert, or even a student, to “tell me what you know” about some topic. But now I’m thinking it makes more sense to ask people to tell us what they don’t know.

Why? Consider your understanding of a particular topic to be divided into three parts:
1. What you know.
2. What you don’t know.
3. What you don’t know you don’t know.

If you ask someone about 1, you get some sense of the boundary between 1 and 2.

But if you ask someone about 2, you implicitly get a lot of 1, you get a sense of the boundary between 1 and 2, and you get a sense of the boundary between 2 and 3.

As my very rational friend Ginger says: More information is good.

15 thoughts on “Tell me what you don’t know

  1. This is my single biggest problem with the way so much operates: people hide what they don’t know and they toward dense jargon as a method of obfuscation. I think this process is internalized and operates mostly at a partially conscious level.

  2. Don Rumsfeld is the authority on this topic :) :

    “Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.”

  3. The following advice is from Lewis Thomas,the great medical educator, theorist, and essayist.

    I suggest that the introductory courses in science, at all levels from grade school through college, be radically revised. Leave the fundamentals, the so-called basics, aside for a while, and concentrate the attention of all students on the things that are not known.
    Lewis Thomas


    • Oooh, I hate Lewis Thomas’s writings, which to me epitomize a showy “humanism” beloved of my high school English teachers. I much prefer Oliver Sacks, who fills his essays with information rather than merely banging on a point over and over.

      • Well, I have read almost nothing by Lewis Thomas. The quote is about the limit of my acquaintance with his work, and I like it and the idea behind it.
        (I have read only one book by Sacks. It was OK but made no great impression on me)

  4. The stuff you don’t know you don’t know, is either your mistakes with the stuff you do know, or stuff you don’t even have a pathway to knowing. Usually, the stuff you don’t know you don’t know is what you need to ask others about.

  5. I slightly disagree: I think asking about 1) also helps with calibrating 3):

    a) They may not know they don’t know something, but nobody knows. That gives little information.

    b) They be making an implicit assumption without thinking about, but you know and can tell them.

    c) (This is the case where asking 1) helps), they may explicitly say they know something, but you have strong evidence they are wrong. That is good information, especially if you give them the evidence and it is rejected.

    Example: software startup seeks funding from Sand Hill Road venture capital firm.
    VC: who’s your competition?
    1) We know of X and Y, but we’re doing something different.
    2) There might be somebody back in Boston or Hew York doing this, but we don’t know well enough.

    3) Of course, the VC may have seen a similar proposal a week before, or even be far along in funding somebody.*
    If VC asks startup:
    How sure are you that X and Y are the only competitors around here?

    Good answer: we’ve looked, couldn’t find any, but from long experience there might be stealth startups or internal efforts, but we’ve already done good development on a shoestring, have track record for good execution, etc.
    Bad answer: we’ve looked hard and we are absolutely certain no one else is doing this.

    * This happens quite often.

  6. The real nuance here is what does it mean to know? It can only be understood in context of relative comparisons, is and can never be static (as what we think we know changes across time and tasks, heck just being asked any of the three has different certainity confidence effects from asking), and both the asker and respondent may suffer from a lack of effective communication in conferring why/how they know, not know, or not think they know. Knowing is a collective postulation with subtleties in expression, conference, of degrees of applicability. Thus the answer to the first two is always the upper and lower bound limits of what all parties participating in the exploratory endeavor, with the answer to three likely residing in those who are not participating. Knowing anything belongs to no one of any of us.

  7. You’ve left out a potential 4, “unknown knowns”: things that people assume about a topic, but aren’t consciously aware of assuming, either because some piece of knowledge has become thoroughly internalized, or because that ‘knowledge’ is an unconscious presumption. Asking about 2 is perhaps also likely to reveal the boundaries of 4. (Idea is from Žižek.)

  8. Pingback: Handy statistical lexicon - Statistical Modeling, Causal Inference, and Social Science

Leave a Reply

Your email address will not be published. Required fields are marked *