Skip to content
 

Thomas Basbøll will like this post (analogy between common—indeed, inevitable—mistakes in drawing, and inevitable mistakes in statistical reasoning).

There’s a saying in art that you have to draw things the way they look, not the way they are.

This reminds me of an important but rarely stated principle in statistical reasoning, the distinction between evidence and truth.

The classic error of novices when drawing is to draw essences—for example, drawing a head as a pair of eyes and a nose and a mouth and a couple of ears and a chin etc. The mistake is to draw linguistically rather than visually. It’s easy to recognize this error but hard to fix. If I, as an unskilled draftsman, try to draw visually, I don’t do a good job. I think the only way I could really do this is by cheating and putting a grid across my field of view and a grid on the paper I’m drawing on, and fill in one little grid square at a time.

That said, there are tricks to teach novices how to draw, and the tricks involve constructing an image from essences, but essences that are geometrical rather than real. So you construct a dog picture, say, from a set of circles and boxes. The idea is that it’s so hard not to draw based on essences, that the best step toward truly visual drawing is to use abstract geometric essences. A related trick is to observe and then draw the negative space, or to draw things upside down; again, these are methods for detaching you from your preconceptions. To put it another way, you can’t draw “linguistically” if you stop yourself from “reading” the face as a mouth plus a nose plus etc.

Now let’s move to statistical reasoning. It’s my impression that applied researchers are working with truth (as they perceive it), not evidence. Or, to put it another way, when they try to “draw the picture” of their evidence, they do it by putting together pieces of truth: This effect is real, that effect is zero, etc.

The question, then, is how to help people. By analogy to drawing, it’s not enough to simply tell people to summarize the data without preconceptions (or, to be more precise from a statistical perspective, to express their preconceptions formally within the statistical model): it’s just too hard for novices to do this from scratch, in the same way that “Just draw what you see” is advice that’s too hard for civilians like me to follow while drawing.

So what we need is a set of tools that will allow people to summarize the data they way they look, without getting tangled in essences. My usual recommendation is to display everything (as in figure 3 of this paper) rather than pulling out statistically significant things to tell a story. For an example of what not to do, see the article discussed in section 2.2 of this paper. It’s hard to learn from data when you’re already telling the story you want to see, in the same way that it’s hard to draw a dog if you see it as a collection of existing parts (head, legs, body, tail).

What you need to draw things the way they look, rather than based on your view of essences, is to develop a sort of contextual dissociation. This is the way that drawing can yield new insights rather than just regurging your preconceptions.

Similarly with statistical analysis: you need to go back and forth between your substantive understanding (including your preconceptions) and a more dissociated, data-first, descriptive presentation.

I think more needs to be said and done in this area.

P.S. I wrote this post 6 months ago; just a coincidence that it came up a few days after our recent discussion of coding and drawing. As commenter S wrote, “If we all were 10/10 on everything we would only be able to go down.”

66 Comments

  1. Oh no!

    The drawing discussion comes one more time, now I have to disagree with everyone again =)

    • Mikhail Shubin says:

      I guess the biggest thing I disagree with is the premise:

      “in art that you have to draw things the way they look, not the way they are.”

      I would say the opposite: “draw the things as they are!”

      I wonder where the statement is coming from, how old it is. I was recently reading John Ruskin’s lectures on art from 1870, he did recommenced to draw things as they are. But maybe he was already old-fashioned in 1870.

      But hey, this is a philosophical statement, as all this words are philosophically loaded. “Things…”, “Appearance…”, “Essence…”. Maybe I actually agree with you, we just operate in different terminologies!

      • To be realistic, you have to draw the thing the way the light interacts with it… not the way it is in the abstract.

        If you put a marble statue that’s been roughened up to matte, not polished, in a white room and illuminate it from all sides, it will essentially disappear. you know as it is, that it’s contoured, but you can’t see the contours when the light comes from all around. It’s the shadows and things that give the statue its 3D look.

        Another good example is “Vantablack” the SUPER dark pigment:

        The wikipedia page has a picture of a crumpled up piece of aluminum foil covered in vantablack… it looks perfectly flat.

        https://en.wikipedia.org/wiki/Vantablack

        • Martha (Smith) says:

          Wow. Never knew about this stuff before.

        • Mikhail Shubin says:

          I need to add that during the world history of art, drawing superblack pigment was not very popular.

          The main focus of classical western art is on drawing humans. And the classical approach goes like this: to draw a human you need to understand how to draw a naked human first. To draw naked humans, you need to learn how the skeleton works, how muscles are attached to the bones. You lean it by practicing a lot. You dont see the skeletal bones under the clothes, but (according to the classical school) it is essential you know them. You dont draw what you see, you draw how things are.

          But then there was Impressionism (which is about impression), and them in around 1900 artist learned different tricks of how to draw dill twilight or a very bright sunlight. These tricks are, indeed, based on knowing how human perception works. And then there were non figurative art..

          Or look at the stylisation. Look how rocks are depicted on ortodox icons. They are neither “how things are” (the are not geologically correct) not “how things are seen” (rocks dont look this way). But it does not makes them bad, right?

      • Here’s a nice article about Vantablack where they put it on one of two bronze busts… look about halfway down the page:

        https://www.sciencealert.com/this-object-has-been-sprayed-with-the-world-s-blackest-pigment-and-it-s-freaking-us-out

  2. jim says:

    “cheating and putting a grid across my field of view”

    Why is that cheating?

    To me the “graph paper” approach in statistical analysis is figure three of the millennium development paper you cited: show all the data and map out the outcome.

    I don’t think the “drawing” analogy is quite on. When you’re drawing, you can stare at the completed version of what you’re trying to produce. You can’t do that in statistics. You don’t know what the completed version looks like. “Mapping” is a more appropriate visual analogy. In mapping, you can’t see the completed product. You can only access it’s true form through the data you acquire with each step, and you must continually retrace your steps and measure from different angles to ensure accuracy.

    The paper you referenced in your other paper is equivalent to mapping the geology of Alaska with three helicopter stops and no positional cross-checking.

  3. jonathan says:

    There is a limit: you cant eliminate choice or, if you do, then you substitute something that makes the choice for you. I suppose we could have research assigned by a sorting hat, but even there you’re being sorted according to inclinations and talents which group you. There is no way you cant construct an image of future outcome. Even if you decide you’ll consider alien interventions, you’ll still rank that somewhere on a likelihood scale.

    So I completely agree transparency is essential. The absolute worst part of reading nearly all papers are questions like: what are they not saying? Which means, how am I being manipulated?

    I’d love to do away with the pretense level where work is presented as though it has no choice behind it, as if ‘this’ topic was picked only because a grant could be obtained or whatever. I find nearly all papers are reluctant to say what they’re about, as though admitting ‘you’ were looking for a result is bad (outside physics, etc., where saying you were looking for a result for this or that reason is part of the presentation process). I want to know why ‘you’ chose to do this work. This mostly happens outside the paper process, as though the combination of seminars, general knowledge within the field, and internet sharing is the equivalent to including that in the paper. I dont think it is equivalent because there is no ‘judge’ in any learned community, but rather there are loops and threads of people who support and those who dont, either because they disagree or dont care. And that perpetuates bad papers and bad results because the support side will continue and what’s missing clearly gets omitted in repeated tellings.

    We all played the telephone game, right? If there is a general state of ‘knowledge’ about a work, what gets passed on is the telephone game, which means that material needs to be in the paper or it is diluted and re-used inappropriately. A lot of the issues with ‘cockroach ideas’ is they get stripped of the full reality of their wrongness, so they can be recycled. We cant expect journals or whatever to attach ‘general knowledge’, but we maybe can expect statements that reveal the choices and beliefs that motivate the paper so you can at least more easily check how something is being used. There are cases where work is cited for the opposite sense as its intent, so a statement of hope or intent and thus choice reasons could help identify the nature of arguments better. That’s the easy case.

    As an aside – my mind wander, I’m sorry – the first internet relays like this that I remember were text posts shared which purported to be a story or official information, and which included fake references. But you couldnt really check as well then; there was a ‘golden’ period for an almost new form of misinformation, in which you could post material equivalent to a lying WANTED poster and it was relatively hard to check because not much was on line. You had to rely on others to ferret out the truth.

    And there were people – like me – who did that. And now I wonder what the fuss was about. Example: website of a great looking girl who lived in Canada, was deaf, loved knives (not cooking ones), and large penises. Of course it was actually some guy in Kansas. Now, I assume the amount of truth on the internet compared to the amount of untruth is that of a real point to the complex field, meaning it resolves to some truth, but that truth may be that some guy in Kansas is pretending to be whatever that is.

    That is, there was a time, a very short time, when there was an irrational degree of hope that truth would be generally available. The irrationality was not recognizing that truth would disappear by becoming ubiquitous: that any point would be represented as the values of perspectives (thank you postmodernism) so truth becomes more completely threaded to ‘what you believe’, which places that firmly in ‘what do you believe’ structures.

    When I was child, we were driving through to or from Toronto listening to a radio show about editing sound. They sent a tape to some experts with the simple problem: identify where it has been altered. Not one got one edit correct. Like Elvis Costello said, you’re only inches on the reel to reel.

    • Martha (Smith) says:

      Jonathan said, (edited to include just the points I think are most important)

      “There is a limit: you cant eliminate choice or, if you do, then you substitute something that makes the choice for you….nearly all papers are questions like: what are they not saying? …

      We all played the telephone game, right? If there is a general state of ‘knowledge’ about a work, what gets passed on is the telephone game, which means that material needs to be in the paper or it is diluted and re-used inappropriately. A lot of the issues with ‘cockroach ideas’ is they get stripped of the full reality of their wrongness, so they can be recycled. We cant expect journals or whatever to attach ‘general knowledge’, but we maybe can expect statements that reveal the choices and beliefs that motivate the paper so you can at least more easily check how something is being used. There are cases where work is cited for the opposite sense as its intent, so a statement of hope or intent and thus choice reasons could help identify the nature of arguments better. That’s the easy case.

      When I was child, we were driving through to or from Toronto listening to a radio show about editing sound. They sent a tape to some experts with the simple problem: identify where it has been altered. Not one got one edit correct.”

  4. I’m not quite sure how this fits into the whole conversation, but it’s an interesting thing to some people, so I’ll put it here.

    Throughout my life I’ve heard people say to “close your eyes and visualize X” as a method of helping people try to understand X.

    It took me until I was in high school to realize that this was a meaningful thing for some (most) people. My brain **does not do that** like **at all**. This guy says it already so you might as well click the link to understand what is Aphantasia The interesting thing is that some people never figure this out until much much later than my say ~15 year old revelation. For example the above was written by a guy age 30. The existence of the internet has created groups of people who get together and discuss their differences from “normal” https://www.reddit.com/r/Aphantasia/. An interesting component of this for me is that I have a very developed abstract sense of space (ie. of multi-dimensional position). No, I can’t “visualize a 35 dimensional space” or anything like that (I can’t visualize anything) but I can absolutely understand points in space and their relative position, and projections of points in 3D onto planes and whether things can be rotated to match other things, stuff like that… all without *any* sense of visualness (ie. no color, transparency, shinyness… instead it’s like you might imagine a blind sculptor who creates a sculpture that “feels” right, though there’s not really a tactile component either)

    I’ve also heard about people who have no inner monologue. They just don’t use words inside their head to think thoughts. I think that’s really really WEIRD because as a person with Aphantasia, if there were no images, and no words, there just wouldn’t be any thought at all. Most of the people whose story i’ve read say that they technically *can* create inner monologue but its exhausting and unnatural and they just don’t do it with any regularity. I’m pretty sure most of these people are normal visualizers. Visualization is a spectrum, from near zero (Aphantasia) to near 1 (Typical) to above 1 (Hyperphantasia). My wife is probably about a 1.2 to 1.5 on that scale.

    The brain is a weird place. But it makes me think that there are reasons why people here at this blog tend to “get it” when it comes to research and many people, even people in *science* don’t get it. And when I say get it, it has to do with a variety of things:

    1) the difference between the model and the conclusions from the model vs the reality.
    2) the importance of the details of the calculations and the numbers. A sense of when precision is important vs not, not getting hung up on the details when the big picture is at stake, and not getting hung up on the big picture when the details refute it…
    3) the language needed to be precise about what your conclusions mean
    4) expressing ideas into formalized language / symbols / calculations
    5) understanding and intuiting logical argument
    6) understanding models as a process

    There are probably a number of other things I should mention but didn’t think of in the above list.

    The thing is, I think some people naturally have the facility to “get it” when it comes to science: constructing logical argument from data using symbolic analysis, math, visualizations, simulations, etc vs other people who are always going to struggle with this. And it’s clearly a spectrum like visualization is a spectrum.

    My point is *most definitely not* to say that “if you don’t got it, then don’t even try”. I mean, tons of people are never going to be NBA players, but they still enjoy pick-up basketball games for example. Tons of people are never going to be the visual artists that get called up by movie studios to create matte paintings, but they still enjoy drawing or painting.

    What I do think though is that if you struggle with research and understanding how to frame your question into a data driven model and evaluate that model, and figure out how to compare models, and express the models into symbolic form in a computer language, and work with computer languages to manipulate data to check the validity, and recode things that are missing, and decide which graphs to make and how to figure out what the graphs mean after you’ve seen them, and whether you have strong evidence or weak evidence in your data…. Then maybe accept that this is one area where your brain is going to struggle, and just embrace the struggle, meaning:

    1) Work hard at it, and expect it to be hard work. It’s hard even for the experts. Andrew says “Statistics is hard” almost daily for example.
    2) Get HELP from people who do it more easily
    3) Be humble about the idea that these things don’t come naturally to most people
    4) Be proud of the fact that you care enough to get help and get closer to right

    For those of us who do something more fluently (whether that’s visual arts, or symbolic calculations, or mathematics or whatever) DON’T assume others have your ease and fluency, some things you take absolutely for granted are *literally impossible* for others (remember, I can’t visualize ANYTHING any no amount of encouragement will help, it’s like a person without legs can’t walk). Usually those people have some other skills, so in collaborations it’s great to emphasize complimentary skills. I have a friend who loves to tweak things until they look good, fonts, colors, icons, text alignment, kerning, etc I totally appreciate it when they do that for me.

    Anyway, there you have it. Maybe the connection between drawing and statistics is really just that both require certain skills that come more naturally to certain people and that are associated with brain structure that makes it easier for certain people to do certain things… So some people have to work harder than others.

    my 2c

    • Keith O’Rourke says:

      Daniel: Hope you don’t mind if I ask if you can process and understand diagrams (say on paper) without having to translate into syntax of some sort?

      • I don’t mind at all, anything I can actually see with regular vision I process more or less normally, it’s really just with my eyes closed all there is is black, or maybe some light filtering through my eyelids… there’s no simulated vision.

        yes I dream more or less normally.

        also I tend to memorize things by converting them to verbal syntax… like when I’d take my kids to the zoo when they were little, I would either take a picture before we entered or memorize a phrase about what they were wearing.

    • Martha (Smith) says:

      Wow. Kinda mind boggling. Too much to comment on completely, but here are some comments:

      You said, ” (I can’t visualize anything) but I can absolutely understand points in space and their relative position, and projections of points in 3D onto planes and whether things can be rotated to match other things, stuff like that… all without *any* sense of visualness (ie. no color, transparency, shinyness… instead it’s like you might imagine a blind sculptor who creates a sculpture that “feels” right, though there’s not really a tactile component either)”

      That’s quite different from me. I find it hard to imagine how you can understand “projections of point in 3D onto planes, and whether things can be rotated to match other things” without any sense of visualness. But another thing that I don’t understand is that you seem to equate “without any sense of visualness” with “no color, transparency, shinyness…” To me, things like color, transparency, shininess are not crucial to visualizing something. I suppose there are some situations where these things are part of visualizing, but I think of them as things that might sometimes be relevant, but that are not relevant for things like projections and rotations. But something you don’t mention is kinesthetic sense. To me, that is very relevant to rotations and reflections (as well as to lots of other things).

      • Andrew says:

        Martha:

        I’m very strong at visualizing in three dimensions (take that, James Watson) but I don’t think I can really visualize color. I’m not colorblind or anything like that, and I use color in my visualizations, but when my eyes are closed and I try to conjure up color (for example imagining a red sun setting in a blue sky), I’m picturing the sun and the sky, but the colors are just labels.

        And I know of people who can’t visualize anything. Not even a dot.

        • Martha (Smith) says:

          Andrew said,
          “when my eyes are closed and I try to conjure up color (for example imagining a red sun setting in a blue sky), I’m picturing the sun and the sky, but the colors are just labels.”

          I tried this, and couldn’t get the colors visually — I just had the words for them. BUT — I often have spontaneous/unintentional visualizations (e.g., memories) that have colors. (In fact, while and since writing the previous sentence, a couple of such technicolor visual images have popped into my mind.)

          • Andrew says:

            All this discussion makes me miss Oliver Sacks!

            • Phil says:

              Missing Oliver Sacks, yes. Andrew, I remember you and I went to see him talk at UCB about 25 years ago! At that time I was interested in a lot of stuff about how our minds work and how we could make them work better, so I was reading stuff by Oliver Sacks, and Massimo Piatelli-Palmarin (is that the right spelling? If only I had some readily available, searchable knowledge repository that I could use to check), and Stephen Pinker back when he was still writing about things he actually knew about, and VS Ramachandran, and that got me into some stuff about animal intelligence and such: Jane Goodall and Berndt Heinrich and more recently Jennifer Ackerman. It seems like everything I read has surprising information and fascinating stories.

              Daniel raises the issue of ‘visualization’. I think I do it about like most people do it, but in my case (and I have always assumed this applies to others) it is not purely ‘visual’. If someone tells me to visualize a cow I can ‘visualize’ a cow, but if they then ask ‘what color is it, and does it have any blotches on it’ I realize I haven’t really pictured a cow, what I’ve got is a mental image of an idealized cow. I can put color on it, sure, but it’s not automatically there, whereas of course if I looked at a real cow my ‘image’ would already include that.

              • This probably makes you about a 0.8 on my scale above… so fairly normal, maybe a tiny bit below avg. my wife will start telling you what the splotches on its nose look like… like I said she’s probably a 1.3 or something. My impression is many good visual artists are maybe a 2-3, they can visualize the light glinting off the surface of the wine bottle or the iridescence of a beetle with their eyes closed etc.

                I’m like 0.07 or something.

              • > stuff about animal intelligence and such
                Cool – I first learned about experiments years before I took my first stats course in both philosophy of language and semiotics course. In philosophy it was think like experiment to support linguistic relativity and in semiotics given animal communication was a big interest experiments about claims of mainly chimpanzee language use. In both the experiment were dissected and critically assessed. At the time, they were all horribly flawed with apparent serious investigator bias to believe the results. Probably a very good way to learn about experiments at least until sampling variation becomes the bigger issue.

                As an aside, semiotics is the field that tries to formalize and study most of the topics being raised here. For instance, Peirce’s claim that one can prescind colour from space but not space from colour (your cow example). http://www.commens.org/encyclopedia/article/gava-gabriele-prescission

    • jim says:

      I learned only a little while ago that there are people who can’t picture things in their minds! Shocker!!

      I can picture things in my mind as clear as a video, but it’s discontinuous. I can replay scenes from a movie in my mind – I just tried that out with the biplane crash scene from 1917 – but I can’t remember every detail. Color too, although that scene doesn’t have much color. A few months ago I watched Band of Brothers again, but I’ve never forgotten the scene where Penkala and Skip Muck were killed by a direct hit on their foxhole at Bastogne. That scene will replay in my mind a thousand times over the rest of my life.
      How do other people remember scenes from movies with no ability to visualize?

      I can have a full conversation in my head too, no problem. Bizarre that people can’t do that!

      • Andrew says:

        Jim:

        I have an excellent memory for events but I’m very bad at remembering exact lines from dialogue (real-life or on screen) or song lyrics. Also, I love to listen to music and I can recognize all sorts of things I’ve heard over the years, but it’s hard for me to hum more than about four bars of anything. I just can’t hold it in my mind. (Also, I’ve never learned to read music so I can’t “cheat” by remembering the notes.)

        My friend Phil has no memory at all. OK, not really. But that’s what he says. And I agree that he has a poor memory. On the plus side for him, he has a much better grip strength than I do. Once I broke my hand and I was going to physical therapy and improving my grip strength, and just for laffs I tried my good hand, and that one was really weak too. I’d never thought about there being natural variation in grip strength! I suddenly understood why I’ve always had difficulty with firm handshakes and throwing the frisbee.

        Overall, I think that human variation is underrated, with the exception of certain easily observed traits such as size, shape, beauty, upper-body strength, speed, flexibility, musical ability, and intelligence.

        • Martha (Smith) says:

          Andrew said,
          “Also, I love to listen to music and I can recognize all sorts of things I’ve heard over the years, but it’s hard for me to hum more than about four bars of anything. I just can’t hold it in my mind. (Also, I’ve never learned to read music so I can’t “cheat” by remembering the notes.)”

          To me the idea of remembering the notes and using that to remember the music seems really bizarre. I’ve never been good at sight singing (which would be necessary to remember the music by remembering the notes), but I can remember tunes if I’ve heard them enough. (However, very often I remember a tune, but not what piece of music it is from.)

          • Martha (Smith) says:

            Above I said,
            “To me the idea of remembering the notes and using that to remember the music seems really bizarre. I’ve never been good at sight singing (which would be necessary to remember the music by remembering the notes), but I can remember tunes if I’ve heard them enough. “

            This morning I thought more about this, and realized that (for me at least) remembering a tune is a kinesthetic process — that is, it is largely “muscle memory” — the muscles in my mouth and throat (and sometimes my abdomen) seem to be well co-ordinated with the sound (both rhythm and musical intervals) that they help produce (I say “help” produce, because there does not seem to be any vocal cord involvement in remembering a melody, although the vocal cords are involved in singing it.)

            Also, I remember the phrase “muscle memory” being used when I was taking voice lessons. I did something wrong, and the teacher said, “OK, let’s do it right several times, to correct your muscle memory.”

            Also relevant: I seem to have a kinesthetic connection with visual things as well as with aural things. For example, when I mentally rotate something in my head, I sense some muscular sensations involved.

            Fun (“Only in Texas!”) fact: When I was taking voice lessons, my teacher once told me how to remember a fourth (in the sense of musical interval): It is the first interval in the song “The Eyes of Texas Are Upon You.” (https://www.youtube.com/watch?v=ioxwwFhMhz4)

            • Joshua says:

              Martha –

              Yes, I think that “muscle memory” is a big component of this. That is largely how I think about learning in general – that the more information or memories are stored with multiple means of connection and access, the easier it is to recreate. When we learn something, we learn it better if we have multiple neural networks connecting to what we’ve learned. Connecting a kinesthetic network makes the information more ‘robust,” in as sense. Of course, individual variation is important – for example some people are better at learning things abstractly without a concrete association – although there’s a fair amount of good evidence that calls into question many of the assumptions we make about “learning styles.”

        • jim says:

          See, now all this stuff would be *very* interesting to study! This is one case where a descriptive study would be so cool, just to characterize the range and variation of all these different kinds and aspects of memory.

          That’s so interesting that you recognize musical patterns but can’t recall them! For me having music to anything makes it much easier to remember.

          Recently I discovered a new (to me) Japanese band and I’m *totally* hooked on their music. Something about the melodies is so…something…I just can’t get them out of my head! I can only say a few of the words, they’re all in Japanese, but the melodies are burned onto my brain.

          Yes, I agree, humans have *so much* variation.

          • Martha (Smith) says:

            Jim said,
            “Something about the melodies is so…something…I just can’t get them out of my head! I can only say a few of the words, they’re all in Japanese, but the melodies are burned onto my brain.”

            Sounds like “ear worms”.

            Jim also said:
            “Yes, I agree, humans have *so much* variation.”

            Yes, yes, and yes. And I believe that recognizing that is important for being a good teacher (and probably for being a good physician, a good writer, etc., etc.)

      • Martha (Smith) says:

        Jim said,
        “I can have a full conversation in my head too, no problem. Bizarre that people can’t do that!”

        Agreed. What I have trouble with is *not* having those conversations-in-my-head. They seem to be my mind’s default unless it’s otherwise fully occupied.

    • Thanks for the Aphantasia link — this is amazing, and I had never heard of it. Not long ago I read an essay by someone who only became “self-aware” in his late teens, which was very bizarre, but unfortunately I can’t find it now.

      You write, “Maybe the connection between drawing and statistics is really just that both require certain skills that come more naturally to certain people and that are associated with brain structure that makes it easier for certain people to do certain things.” The latter is true and fascinating, but as useful as it would be it is a non-starter to address this in teaching.

    • Tom Passin says:

      I am fairly colorblind in red-green. I can see those colors in some circumstances, but the ability is weak. One day I was flying (as in piloting a rented airplane) across a big river. There was a wake below, left by a speedboat. I was almost ecstatic about the beauty of the complex set of ripples side lit by the late afternoon sun.

      My companion was a woman with an exquisitely developed color sense. She saw nothing of interest below. I could hardly believe it. We thought later that I may be more sensitive to patterns of light and shadow because I am weak on color.

    • Carlos Ungil says:

      >This guy says it already so you might as well click the link to understand what is Aphantasia

      Whenever I read about this subject, in the end I don’t know where am I (to the extent that I can imagine things it doesn’t really matter whether my eyes are open or closed).

      All the tests I’ve seen are nearly useless: “How to know if you can visualize things: answer this simple question ‘can you visualize things?'”

      But I can tell what I did today, so I guess I’m fine!

      “I’ve always felt an incomprehensible combination of stupid-smart. I missed a single question on the SATs, yet the easiest conceivable question stumps me: What was it like growing up in Miami?

      I don’t know.

      What were some of your favorite experiences at Facebook?

      I don’t know.

      What did you do today?

      I don’t know. I don’t know what I did today.

      Answering questions like this requires me to “do mental work,” the way you might if you’re struggling to recall what happened in the Battle of Trafalgar. If I haven’t prepared, I can’t begin to answer.”

      • Here’s the one I use on a lot of people:

        Imagine a cow. Give it a little time to sink in…

        now…

        What direction is it facing? What colors does it have? Where is it standing? What is the background? What color? What textures? Where is the light coming from?

        If you don’t have answers to at least half those kinds of questions, you’re probably lower than a 1 on my scale. If you have very specific answers to all of those questions you’re probably well above 1

        If, like me, you have no answers to any of those questions except maybe the direction it’s facing, you’re probably Aphantasic

        • Joshua says:

          My guess is that there could be a significant interaction effect with things like what a person does for a living, which of course partially reflects innate characteristics but perhaps not. Could be a mediating or moderating effect?

          For example, engineers might tend towards identifying one sent of specifics, a gardener another, an artist another, a biologist another, etc.?

        • Martha (Smith) says:

          My imagined cow has its head at the left, but the head is turned to face out from the picture. It’s a light yellowish-tan color. No idea about where it’s standing, nor the background, nor textures, nor where the light is coming. I suppose I could add in those details, but they’re not in the spontaneously generated image.

          • I think this puts you pretty close to median ability, around a 1 on my scale. People with advanced visualization abilities say things like it’s in a green pasture with long tufty grass, there’s lots of yellow flowers among the grass, the cow has horns but the right one has a broken tip, there are rolling hills in the background, it’s mid day, the sun is shining from the left casting a deep shadow on the side of the cows face, the sky is deep blue with big puffy clouds one of which is a thunder-cloud… There are other cows dotting the hillside in the background… etc etc.

            • Martha (Smith) says:

              I think this is a situation where the wording of the question may influence the results to some degree (or at least for some respondents). If you had said, “Imagine a cow with surroundings” (rather than just “Imagine a cow”), I would have imagined surroundings as well as the cow.

              • Yes, but intentionally I make it very simple, because I think the degree to which people do or don’t add those things automatically is indicative of what their personal tendency / fluency is… People who naturally just create all that stuff obviously have much more visualization fluency than others.

      • Mikhail Shubin says:

        Wow, this is interesting!

        This actually explains why Im so bad in learning languages. All of the language courses I have tried to go to split students into groups and ask them to do small talks as a part of a learning process. And I just cant do it, Im struggling in my own language, I cant learn anything under such a stress! Why cant we just learn languages in the old way, by memorising grammar books.

        • Martha (Smith) says:

          I’m glad I learned languages before they split students into groups. To me, an important part of learning a new language was getting the (physical) feel (and sound) of it — I needed a teacher to model that and correct me when I got it wrong. So memorizing seemed the first step to learning the language.

          On the other hand, maybe if I had had the small group stuff, that might have been better for acquiring fluency. Hard to say.

    • Joshua says:

      Daniel –

      > The thing is, I think some people naturally have the facility to “get it” when it comes to science: constructing logical argument from data using symbolic analysis, math, visualizations, simulations, etc vs other people who are always going to struggle with this. And it’s clearly a spectrum like visualization is a spectrum.

      I’m reasonably sure that innate abilities matter here – but I think that experiences and other environmental factors (particularly at developmental stages) matter a great deal as well.

      • Martha (Smith) says:

        +1

        And it may be hard to tell just which things are “innate” and which are learned by early experience and environmental factors. For example, what toys an infant has, or how the child’s caretakers interact with the child at a very early age may make a big difference. (I’m not thinking about just explicit teaching — but also about what the child learns by seeing, feeling, or hearing, and by interacting spontaneously with the environment — which depends on the environment– which could include siblings and interactions with them, as well as many other things.)

      • jim says:

        I get concerned about the use of the term “innate ability”.

        No doubt, there is such a thing. But I think much of what we think is innate is actually learned through practice. For example, if a child has two well educated parents, they speak like well educated people and think like well educated people and they teach the child in that way too starting from day one. So what would be really surprising is if that child showed little or no promise. OTOH, the kid who’s single parent is a high school drop out won’t start getting the training the other child got until highs school at least – a decade of practice lost.

    • Mikhail Shubin says:

      Thanks, this is a very insightful read!

      I guess Im somewhere close to 1 on your visualisation spectre.

      I had a weird experience related to visualisation and dreams. I once dreamed of a landscape: a lake, a small island on a lake, and a single tree on this island. The tree was immensely beautiful. When I waked up I tried to visualise this tree in term of shapes and colours, but it turned out to be impossible. This made me think that what I seen my dream was the concept of beauty itself. The bunch of neurones in my brain that judge “this looks nice” just got exited on their own.

      PS: I dont have an inner dialogue. I have an inner voice, but it does not engage in the conversation, it just repeats the same couple of words again and again for the last 20 years.

      • So if I were to ask you to write a couple of sentences about your breakfast… would you not be able to think those words before you type them in? It’s like they’re pre-verbal until you say them or type them?

        I don’t really understand the non-internal-monologue thing, because basically there are words going through my head literally ALL day long. Everything that happens I kind of talk to myself about. “Gee I’m hungry, what kind of lunch do I want?” kind of stuff.

        • Mikhail Shubin says:

          I do have internal monolog sometimes, but it is very basic. The biggest part of the inner talking I do takes a form of an imaginary conversation between me and someone else. Does not need to be anyone specific, I could imagine me giving a lecture or writing a comment.

          Like when I read your question, I started forming a reply in my mind. And this reply takes a form of a “spoken” sentence which I can “hear” in my mind until I write it down.

          • > And this reply takes a form of a “spoken” sentence which I can “hear” in my mind until I write it down.

            was that supposed to say “I can’t hear” ?

            • Martha (Smith) says:

              Daniel said “was that supposed to say “I can’t hear” ?”

              That surprised me. I interpreted Mikhail’s sentence, “And this reply takes a form of a “spoken” sentence which I can “hear” in my mind until I write it down,” as saying,

              “This reply takes the form of a “spoken” sentence which I can continue to “hear” in my mind until I have the chance to write it down.”

              I guess Mikhail needs to tell us which (if either) interpretation is correct.

              • I interpreted it as either he couldn’t hear it until he wrote it (and read it back) or that he could hear it but when he wrote it down he stopped being able to hear it… but I think maybe your interpretation is probably simplest.

                So then I don’t understand what it means to not have internal monologue because he says he can in fact pretend in his head to be talking… perhaps he just means that it feels very awkward and takes a lot of effort.

              • Mikhail Shubin says:

                What I mean is that after I read Daniel’s question, I start imagining me replying to him. Or, to be precise, I imagine myself replying to someone offscreen. After I write down the answer, I stop imagining this.

                Does this count as internal monolog? Maybe it is, maybe Im just confused with the terminology, this is a new topic for me!

              • I think it counts. But there’s a question of without prompting, do you tend to do the same thing on your own? like do you “hear” words in your head discussing the merits of different solutions to problems you face? like

                “I can’t find the mayonnaise, I wonder if my wife moved it to the smaller refrigerator when she was finding a place to put that chicken we cooked yesterday? or maybe the kids left it out on the table and we will have to throw the whole thing away, that would be too bad, because we just bought it and it’s almost full…”

                etc that kind of stuff goes through my head constantly… similarly if I’m working on a modeling problem I talk through the choices I have.

                my wife on the other hand tends to have flashes of insight as pictures. related things get classified together, then she unpacks her ideas by examining the details… like different icons connected by lines and with circles drawn around them. she’s extraordinarily smart, but she struggles to write text from scratch… half the content of her papers started out as me asking her questions and then writing down summaries of what I thought, whether it’s right or not, which she edits into something expanded and correct.

    • Eric B Rasmusen says:

      I was probably about 50 when I realized people without photographic memory could still visualize things inside their heads. On the other hand, I can play back music very well. But not if it has words. Is there some kind of substitution in brain-space?

      Your thought on people who can’t use words inside their heads is new to me. This explains why some people I know well need to “Talk things out”. Such people need to reason out loud, but they aren’t really talking to anyone else, sometimes to my annoyance. This is a common style of thinking: when such people have a problem, they want to start by talking at someone else, as opposed to my own style of going off to think by myself and then return to consult orally once things get well-formulated. It’s frustrating, but I have one mathematician co-author where I have to realize my main and utterly essential use is as another human being in the room while he talks to himself.

      Getting closer to the post’s main topic: what people like me do is go step by step with visual tasks, filling in first one bit and then another, usually as Gelman explains it in the post by thinking of essential aspects instead of visual ones, but possibly expanding the “seeming” part from one edge to another. I’ve heard some artists are aphantasistic; how do they manage?

  5. Interesting – as you likely know experienced artists paint snow yellow and shadows blue as people don’t actually know what they see (until it is pointed out to them).

    I was trying to get at something similar with “many simply did not realize that statistical thinking and modelling always took place in fake worlds (mathematically) and is transported to our reality to make sense of data we have in hand, that we trying learn from.” here https://statmodeling.stat.columbia.edu/2020/08/05/somethings-do-not-seem-to-spread-easily-the-role-of-simulation-in-statistical-practice-and-perhaps-theory/

    I am trying to use diagrammatical reasoning as a way to make that clearer…

  6. Peter Dorman says:

    I think the analogy to drawing – freeing the eye from the shackles of mental preconceptions – applies to many other realms in addition to statistics. In fact, it has a lot in common with the Taoist notion that I would call creative unlearning. And the strategies used in drawing exemplify the types of discipline that can be used in other contexts too.

  7. Michael Nelson says:

    We design our studies to produce data that will either support or not support our hypothesis. So we necessarily begin with “the truth” (as you put it), then seek evidence for it. Likewise, artists select what to draw based on their impression of it. The difference is that artists are then (as you say) trained to ignore that impression. Social scientists are trained to be an advocate for that impression–we argue for it to our committee, to the grantor, to the reviewers and readers. It’s no wonder we have trouble letting that truth go as we execute our design, conduct our analysis and interpret the results. To do so, we have to overcome the biases ingrained in scientific training and culture–not to mention human nature.

    For example, we’re naturally inclined to believe (and taught) that, because we had this hypothesis before we collected the data, obtaining data consistent with our hypothesis cannot be coincidence. But any set of results will support many different conclusions, and while the a priori nature of our hypothesis lends some amount of weight to our favored conclusion, it does not reduce all the other weights to zero. The weight of our conclusion is lessened when we find the supportive data points through post-hoc data mining, and lessened again if we form/reform our hypothesis based on what the data can support. But again, this is not what we’re taught–or if we are taught this, it’s abstract, and what we see in practice among our mentors and peers is different.

    Andrew, your working solution is to show all the data and I agree, but in the longer term, we need to change how social scientists are trained. A trained artist doesn’t draw a circle for a head, he lightly sketches twenty circles and erases the curves he doesn’t need. Likewise, we need to train scientists by emphasizing the iterative aspect of the scientific method over the adversarial aspect, where an experiment ideally results in a reweighting of possible explanations for observations, not in a single position that must then be defended at all costs.

  8. Leon says:

    To me this analogy is quite direct, in that I mentally convert all kinds of statistical and evidential concepts into plots. To me, plots are to statistics as geometrical essences are to drawing.

    To put it another way, you can’t draw “linguistically” if you stop yourself from “reading” the face as a mouth plus a nose plus etc.

    Did you mean: “*can draw “linguistically””?

  9. I do.

    It reminds me of something Hemingway said about the difference between journalistic and literary writing. Andrew talks about “being tangled up in essences”; Hemingway talked about being “aided by the element of timeliness”. Andrew talks about “preconceptions”; Hemingway talks about “how you were supposed to feel.”

    “I found the greatest difficulty, aside from knowing truly what you really felt, rather than what you were supposed to feel, and had been taught to feel, was to put down what really happened in action; what the actual things were which produced the emotion that you experienced. In writing for a newspaper you told what happened aided by the element of timeliness which gives a certain emotion to any account of something that has happened on that day; but the real thing, the sequence of motion and fact which made the emotion and which would be as valid in a year or in ten years or, with luck and if your stated it purely enough, always, was beyond me and I was working very hard to try to get it.” (Death in the Afternoon, p. 10)

    I applied this to academic writing in this post. See also this one: “How to Imagine Science”.

    And of course this one: “Pictures, Stories, Models”.

    One day, I’ll figure out how to present my intuitions about the constructive tension between narrative and statistical analysis. The way we read “essences” into drawings, and how we’re “supposed to feel” into our stories, serves as a good model for how narratives try to shape statistical analyses.

  10. Thomas says:

    JL Borges wrote a short story, “Del rigor en la ciencia” in which the science of cartography becomes so advanced that only a map of the size of the country is acceptable.
    Maybe he was just poking fun. Or he wanted to draw attention to the gulf between description and understanding.

  11. Many of these comments relate to major issues in cognitive science and cognitive psychology. For years toward the end of the twentieth century, propositional theorists (e.g., Pylyshyn) believed mental imagery was a surface representation (reconstruction) of remembered linguistic (or graph-theoretic) propositions. In other words, we don’t store pictures in our brain. Then along came image-oriented cognitive scientists (e.g., Kosslyn and Shepard) who did studies that made it hard to explain their results with a propositional framework. I believe Shepard’s work on mental rotations, for example, is one of the most significant landmarks in twentieth-century psychology. (Shepard’s data were so perfectly linear that he didn’t need regression to support his conclusions; THAT’s a well-designed experiment.) And Kosslyn’s book, Image and Mind, is a fascinating read for anyone not familiar with this problem.

    As you might tell, I think the imagists prevailed, even though Pylyshyn valiantly wrote rejoinders. However, as several commenters above point out, there is more to the individual differences point of view than many cognitive scientists are willing to accept. Indeed, some people are terrible visualizers and many of the global conclusions of the old cognitive scientists have not held up well.

    That said, I believe that most contemporary visualization research (often done by computer scientists and statisticians) ignores the critical component of memory. Unless a visualizer (human, animal, insect, etc.) can recall the structure of a visualization long enough to react or relate to extrinsic information, I believe the visualization is ineffective. In other words, a visualization that does not make its way through working (short-term) memory to explicit (long-term) memory is basically an unprocessed, momentary perception. And what is that deep structure we’re talking about? Well, read The Grammar of Graphics.

    So what should, in my opinion, “good” research in visualization look like? First, it needs to be well-designed, following all the rules statisticians know about experimenter bias, double-blinding, randomization, etc. Second, it needs to follow the pathway from perception all the way to memory. Bill Cleveland’s important experiments are all about perceptual biases in visualization, but they do not address cognitive biases and other aspects of higher-level processing. How about, for example, designing a visualization that conveys a “story” (to use the lingo du jour among viz people)? Then ask viewers questions involving the elements of the story immediately after viewing and a day or two later. This methodology would resemble what the SAT does with its reading comprehension section.

    Asking questions is a deep problem. It is, in my opinion, worthless to ask subjects questions like “what visualization do you like best?” Or, “On a scale from 1 to 10, how informative do you think this graph is?” Visualization researchers need to read about the methodologies cognitive scientists have used for decades. These involve variables like reaction times, pairwise comparisons, gaze location, distractibility, and so on. And expertise in subjects needs to be controlled. There’s no reason novices can’t be presented with technical visualizations and asked for reactions. The worst subjects, from an experimental psychology point of view, are statistics or computer science grad students. Furthermore, novices can be trained in a real-time experimental environment in such a way that we can measure the slope of their learning curves.

    Alternatively, the visualizations themselves can be presented as the targets of a video game. For example, flash a histogram on the screen (don’t explain to the subject what a histogram is) and then flash a KS value or some other statistic based on the histogram. Ask them to guess the numbers and store the time series of their responses (including reaction times). Reward subjects who achieve higher accuracy scores. AND collect covariate data (gender, age, interests, etc.) to allow some retrospective analysis of individual differences.

    In my opinion, we have only just begun to understand the pathway from perception to cognition in visualization. Until we do, through careful experimentation, we will persist in broadcasting silly (and false) maxims like “less is more.” And those who say pie charts are bad (most computer scientists and statisticians I know), need to read the experimental research papers on this topic (especially those by Ian Spence and Reid Hastie) that firmly demonstrate the falseness of that generalization. Opinions, especially those expressed by astute novelists and artists, are invaluable for stimulating ideas. But they are no substitute for experimentation.

  12. Eric B Rasmusen says:

    People like to dump on Myer-Briggs, but I find it a highly useful theory, and it applies here. My favorite of its dimensions is N/S, Intuitive/Sensing. Some people Sense the world, taking it as it is on the surface, the world of seeming, and other people iNtuit it, filtering it through a framework. The cute thing about the theory is that the Intuiters aren’t Thinkers, they’re a combination of Poets and Scientists, not people who have theories, but people who instinctively try to get to the world of being, not seeming.

    Now think of data analysis. What the post is saying is that the ideal is to be a Senser first, who can take the sample and organize it in some helpful way without imposing any framework on it (e.g. a histogram of one-dimensional discrete data). Or maybe have a Senser on your team. Because, though this maybe isn’t in the post, the essential second step is to pivot and be an Intuiter and reorganize the sample to highlight what’s really going on in the population, totally framing it using a theory (though that includes highlighting the parts of it that don’t fit the theory; whenever you honestly frame something, you inevitably show how some of the datapoints end up outside the frame instead of inside it).

Leave a Reply to Andrew