Skip to content
 

Is Matthew Walker’s “Why We Sleep” Riddled with Scientific and Factual Errors?

Asher Meir points to this hilarious post by Alexey Guzey entitled, Matthew Walker’s “Why We Sleep” Is Riddled with Scientific and Factual Errors.

Just to start with, the post has a wonderful descriptive title. And the laffs start right away:

Positively Nabokovian, I’d say. I mean it. The above table of contents makes me want to read more.

I’ve not read Walker’s book and I don’t know anything about sleep research, so I won’t try to judge Guzey’s claims. I read through and I found Guzey’s arguments to be persuasive, but, hey, I’m easily persuaded.

I’d be happy to read a followup article by Michael Walker, “Alexey Guzey’s ‘Matthew Walker’s “Why We Sleep” Is Riddled with Scientific and Factual Errors’ Is Riddled with Scientific and Factual Errors.” That (hypothetical) post could completely turn me around! Then, of course, I’d be waiting for Guzey’s reply, “Michael Walker’s ‘Alexey Guzey’s “Matthew Walker’s ‘Why We Sleep’ Is Riddled with Scientific and Factual Errors” Is Riddled with Scientific and Factual Errors’ Is Riddled with Scientific and Factual Errors.” At that point, I’d probably have heard enough to have formed a firm opinion. Right now, the ball is totally in Walker’s court.

After reading various sparkly, hard-hitting bits of Guzey’s post, I was gonna say, “Get this man [Walker] a Ted talk!” But apparently he does have a Ted talk. From Guzey:

I haven’t had this much fun reading something online since the days when Gawker was publishing.

Anyway, it seems that Walker’s Ted talk is called “Sleep is Your Superpower.” Your superpower. How Ted can you get?? The modern world is really wasted on us, without Veronica Geng to mock it for us.

Guzey reports, “Matthew Walker is a professor of neuroscience and psychology at the University of California, Berkeley, where he also leads the Center for Human Sleep Science. . . . His book Why We Sleep . . . was praised by the New York Times . . . was named one of NPR’s favorite books of 2017 . . .”

OK: neuroscience professor at major university, praised by NYT and NPR . . . and he even published an article in the Lancet! We’re touching all the bases here. What next, a collaboration with Dr. Anil Potti?

Guzey continues: “A month after the book’s publication, he became a sleep scientist at Google.”

I’ll have to say, if it’s really true what Guzey says that Walker’s book is riddled with errors, there’s something satisfying about hearing that Walker got a job at Google. We have this image of Google as some sort of juggernaut. It’s good to hear that they can make mistakes too, just like scientific journals, universities, news outlets, etc.

Let me tell you a story. I went to graduate school at Harvard. Finest university in the world. My first day in a Harvard class, I was sitting with rapt attention, learning all sorts of interesting and important things (for reals; it was an amazing class that motivated me to become a statistician), sitting at one of those chairs with a desk attached to it, you know, the kind of chair where the desk part flips up so it’s in front of you, and, on the bottom of that desk was a wad of gum.

Back when I was in junior high, gum was almost a form of currency. I’d buy a pack of grape Bubble Yum for a quarter at the corner store on the way to school, then chew it in the morning during the endless hours between first period and lunch. I’d put one piece of gum in my mouth, chew it until it lost all its flavor, then add the second piece, chew it etc., and continue until I had a massive wad, all five pieces, ultimately flavorless, and I’d chew and chew and blow huge bubbles when the teacher wasn’t looking.

I’m not trying to make myself out into some big rebel here; the point is, we all did that. So of course there was yucky gum under all the desks. You knew to never run your hands under a desk, cos you never knew what might turn up. That was junior high.

Then in high school, everyone was much more mature, a lot less gum chewing . . . but still, gum under the desks. I took classes at the University of Maryland, a fine university with an OK basketball team . . . still, they had gum. Then I went to MIT, one of the finest engineering schools in the world . . . yup, gum. But Harvard? I’d hoped Harvard was better than that. But it wasn’t.

Anyway, that’s how I felt, learning that this purveyor of (possibly) horribly false claims is not just a professor of neuroscience at a top university—we know that top universities have lots of frauds—but was hired by Google. Google! Here I am, almost sixty years old (I don’t feel close to 60, but that’s my problem, not yours), and still there’s room for disillusionment.

Anyway . . . the next question (again conditional on Guzey’s statement that Walker’s book is riddled with errors) is, where does Walker fit in here? Did he not fully read the sources he cited in his book? Did he write much of the book from memory, and just misremember a lot of things? Does he think that it’s ok to get the facts wrong in service of a larger truth? Is he just confused? Maybe he didn’t write the book himself, perhaps he had an army of incompetent research assistants?

Just as an example, here’s the first item in Guzey’s post:

OK, I haven’t followed all the links and read the linked studies. That’s not my job—as Bill James once wrote, I’m not a public utility. If you want to read the linked studies, go for it!

The question is: what was Walker thinking when he wrote things like, “the shorter your sleep, the shorter your life span” or “Routinely sleeping less than six or seven hours a night demolishes your immune system, more than doubling your risk of cancer”? I guess someone will have to ask him. I wonder about this a lot, when people write things that are not supported by the data. It’s my impression that people often write things that sound good, without always thinking about their literal meanings.

Here’s another one:

What was Walker thinking when he wrote, “every species studied to date sleeps”?? Did he forget what he’d read in that 638-page book? Did he read the passage in question so quickly that he took from it the opposite impression? Perhaps he was working from memory, and forgot some things? I have no idea.

OK, I can’t resist. Here’s another:

Again, maybe I’m missing the whole point. Walker should feel free to take a few moments from his jobs at Berkeley and Google to respond in comments to explain how Guzey is misrepresenting the evidence, or if there’s something else going on. I’m open to the possibility.

Or if Walker did get all these things wrong . . . then maybe he can write something to thank Guzey for pointing out the errors. I make mistakes all the time—it happens!—and when I learn about them, I feel bad. Maybe Walker will learn about this and feel bad and figure out how to do better next time. I don’t think Brian Wansink ever got around to thanking people for finding all the errors in his papers, but I think he should’ve.

In the meantime, I appreciate that Guzey wrote his post directly and didn’t bother being polite. Politeness is fine, but it has a cost. First, a polite version of the post would be less fun to read, and less fun means I’d be less likely to read it, which would be a shame, given that this is an important topic. Second, I’m guessing that rewriting this post more politely would take a lot of effort on Guzey’s part, which would mean that maybe he wouldn’t have written it at all, or he would’ve included less information, or that he’d have less time to do other stuff. And that would be unfortunate. Guzey’s time is valuable too.

I wouldn’t’ve wanted Mark Twain, Veronica Geng, or David Sedaris to have worried too much about politeness either. I’m sure Guzey makes mistakes too. That’s ok. Mistakes are mistakes, even when written politely.

And, if it turns out that Guzey got it all wrong regarding Walker’s book, then fine, I’ll report back and update this post accordingly. I wouldn’t be the first person to get fooled by something on the internet.

P.S. Personally, I looove sleep. I sleep 9 or 10 hours a night. OK, I don’t average 9 or 10 hours a night. My average has got to be less than 8, maybe even less than 7, because when I get too little sleep for a night or two, it’s not like I balance it with 11 hours the next night. What I’m saying is that 9 or 10 hours makes me comfortable: it’s how much sleep I’d get if I had no kids and never had an early-morning deadline. Based on my observations of others, I’m guessing I’ll need less sleep as I get older.

P.P.S. Some searching also turned up this amusing exchange:

To be fair, though, my colleagues and I once wrote a book with the subtitle, “Why Americans Vote the Way They Do,” but we never actually answered that question! In the book, all our empirical claims were carefully sourced and explained, but then we just said something silly the title. So don’t trust titles.

P.P.P.S. I had so much fun writing this post, and I’m such a damn perfectionist, I spent 2 hours on it! And now I’m getting to sleep really late, I’ll only get to sleep 7.5 hours at most. So I’ll be uncomfortable and tired in the morning (and I don’t even drink coffee! I never touch the stuff, just don’t like the smell and taste), and it’ll probably lower my productivity tomorrow. Damn. I do feel, though, that if I write what I write when I want to write it, eventually I’ll end up writing the things I should write, the way I want to write them.

42 Comments

  1. jim says:

    If it’s about 3rd world unfairness, NPR would eat that right up.

  2. Garrett says:

    “Second, I’m guessing that rewriting this post more politely would take a lot of effort on Guzey’s part, which would mean that maybe he wouldn’t have written it at all, or he would’ve included less information, or that he’d have less time to do other stuff.”

    Guzey actually addresses that:

    “13. Appendix: “why did you only check Chapter 1?”
    I spent more than 130 hours over the last 2 months researching and writing this essay (~5 hours to write the outline; ~60 hours to get to the first draft; ~65 hours to edit and fact-check), which constituted essentially all of my surplus free time over this time period. Continuing at the same pace, it would take me more than 3,000 hours to check the entire book. 3,000 hours is the equivalent of 75 weeks or 1.4 years of full-time work.

    I hope that going through one full chapter, rather than cherry-picking stuff from across the book, demonstrated the density of errors in the book.”

    The book was apparently so bad that he could only get through the first chapter’s inaccuracies after two months. Imagine if he had to be polite about it!

  3. jd says:

    I enjoyed reading this post. My favorite part was the story about the bubble gum under the desk at Harvard.

  4. Terry says:

    “I appreciate that Guzey wrote his post directly and didn’t bother being polite.”

    I found Guzey’s article to be quite polite. There were no gratuitous insults or snarkiness, only straight-forward declarative sentences. When he says things like “This is false”, he is simply saying something true in the most straightforward and unadorned way possible. He also includes many caveats which make the text sound very careful, and makes the blunt “This is false” statements that much more powerful.

    I find this style of writing very persuasive. Nothing appears to be hidden or gratuitous. It makes his statements that much harder to counter with obfuscation or misdirection.

  5. Jonathan says:

    My takeaway is another confirmation that very (?) intelligent people, particularly those with significant mathematical bents, tend to stuff gum in their mouths. I have an entire theory based on this, but no TED Talk. The gist is that sticks of gum are groups which are manipulated – masticated, if you prefer – to extract either the maximum density of flavor or the maximum length of flavor. The spatial ability to perceive the taste content of a stick also includes its perceived longevity as a ‘chew’, as a sort of whale skin you work over (do like the Inuit do). And thus the trait is associated not only with spatial manipulations – sticks of gum into mouth, into shapes, etc. – but with attribute manipulations. I think it would be perfect for TED.

    I also note essentially all the respondents – my n is maybe 10 – blow bubbles. This is done whether the gum is typically bubblegum or just gum. As in, you can blow bubbles with Dentene but it takes concentration and the failure rate is high. This is yet another evaluative skill, this time involving analysis of the tensile strength of the gum, something you can generally measure in the gum’s grain.

  6. Elvis says:

    Long-ish time reader, first time commenter here. Love this, Andrew. Funny story and I can feel your excitement sharing it. Being in the beginning of my career, I especially appreciate the insights on how you, who is someone I really admire, work and think about work.

  7. Phil says:

    Since when did 54 become ‘almost 60’?

    • jim says:

      If you hate running like I do but you force yourself to run for, say, an hour, 54 min is definitely “almost” 60, especially compared to 50, 48, 45, 40 ….

    • Martha (Smith) says:

      Phil Said, “Since when did 54 become ‘almost 60’?”

      +1 — like saying 90 is almost 100.
      .

    • This is humorous from a Physicist ;-)

      • Phil says:

        Yeah. When a mathematician says two quantities are equal, they mean they’re exactly equal. When an experimental physicist says two quantities are equal, they mean they’re equal within 10% or so. When a theoretical physicist says two quantities are equal, they’re within about a factor of 10. And when a cosmologist says two quantities are equal, they mean they have the same units.

        • People routinely fail to understand me when I say “these two quantities are basically the same”, you know like $3250.15 and $2920.12 are both basically $3000

          I once had a guy utterly flame me when I said that around 20 standard drinks in a day would kill you… He absolutely insisted that was the most ridiculous thing he’d ever heard, and he’d routinely had more than that in college…

          of course what I meant by that was that the LD50 for vodka shots all at once is about 13, so we can’t say that 20 drinks is in any way “safe”, and since the effectiveness of the liver ranges over about a factor of 3 across the population, and different people have different consumption patterns, even if 20 in a day is near the median in the population, you’d expect to see the occasional person who could drink 40 or so, so as long as DrinksThatWouldKillYouPersonallyIn24Hrs/20 has a distribution with a median around 1 my claim was correct.

          As a philosophy major he didn’t really understand this…

          • I guess this is just further evidence that embracing uncertainty and variation is pretty far from the norm even among PhD educated people.

          • Jim the Baleen Whale says:

            I’m not sure if I’m missing the joke here, still kind of groggy and having my first cup of coffee, but I’d say how much alcohol one can* drink during a 24 hour period is drastically different question from how much alcohol one can consume at once. Having been a professional musician I have vast amounts of empirical evidence regarding the former question… “when you’re ridin’ 16 hours and there’s nothing much to do”…

            *I will not be qualifying “can” here.

            • Not a joke, just a fact that with differing consumption patterns, differing liver effectiveness, different health status and soforth, the quantity of alcohol in standard drinks that will kill you is well modeled by a Bayesian distribution like a gamma(4,3/20) which has a peak around 20 and falls off in a tail out to 50 or 60 or so. Even 1 drink might kill you if you’ve taken some medications by surpressing your breathing… so variation is an essential part of this model. you can’t falsify the model by exhibiting an individual who had 28 or 34 drinks in the last 24 hours. for every person like that there’s some person with a defective enzyme or a sensitive brain who wouldn’t even try to have more than single digits

              This idea seems to be particularly odd to the high risk group who are most insensitive to the effects of alcohol. Heavy drinkers tend to stay up late hours and manage their blood alcohol as if they had an IV drip. 12-15 beers a night is routine for this population, but they are quite rare making up 1-2% of the population or so.

              • Jim the Baleen Whale says:

                Ah, gotcha. I’m not against any of that, obviously there’s variation; my point was that the question of “how many drinks will kill you” is next to meaningless without taking time into account. An issue which was touched upon a post or two ago on this blog?

              • Yes, one source of variation is the pattern of consumption. Chugging vodka is FAR more dangerous than having one beer every hour on a schedule. The original discussion was about consumption in 1 day. But as you say things vary in terms of how people would usually do that. There are sort of two main patterns of heavy drinking: binging at a party, and long term management. The long term management people are usually people that do this stuff regularly and have learned how to maximize the time where they are at their preferred fairly high BAC… maybe 0.1% or whatever… these are the most severe alcoholics.

                the binge drinkers at college parties or whatnot would be in much bigger danger consuming the same amount but in a shorter time like 4 or 5 hours. So that all goes into the risk as well.

                This is a topic I find interesting, because it has a LOT of statistical challenges and it has mechanistic modeling components, and it is a serious public health issue, so we have had related discussions here on the blog probably a few times over the last several years. I actually have a bunch of research on this topic from a year back or so that I really should write up… it’s not my main area of expertise though, so in many ways I don’t know how best to go about publishing it.

        • Martha (Smith) says:

          “And when a cosmologist says two quantities are equal, they mean they have the same units”

          Equality of units! Having different units would be a form of inequality!

  8. DC says:

    This was a delight to read, thanks Andrew.

  9. Gordon Fox says:

    Re “Every species studied to date sleeps,” I wouldn’t accept that from an Intro Biology student, and I’ve taught it a few times. Most species aren’t vertebrate animals – they’re invertebrates, plants, fungi, bacteria, and so on. Things that anyone interested in sleep studies would have to say “no way you could even define sleep in this critter.”

    Thanks for the entertaining read!

  10. jim says:

    I just googled “wierd (sic) things about how animals sleep” and found this: https://www.countryliving.com/uk/wildlife/countryside/g27810374/animal-sleeping-habits/

    Wow! I hope Walker knows these TED gems about animals!

    According to the link, our nearest relatives, the chimps, sleep on average 9 hours a day. “Paleosleep”! He could have measured their testicles to find out if sleep affects chimp testicles in the same way it affects human testicles. Perhaps, if he dreamed up enough testicle properties to measure, he could have ensured a successfully deployment of Mr. P for a revolutionary insight about testicles and sleep! TED stuff! OTOH, it might be difficult to directly measure these organs, so perhaps it would be easier just to Wansink it. I mean, why leave it to chance to pick which chimp testicle property gets a P-pass, when you can engineer the data and choose the most interesting! I’m envisioning the Wansinkesque experimental apparatus (recorded in the distance in a single grainy photo): a long telescope with giant calipers glued on one end.

    I can already hear Shanker Vedantam announcing the unexpected results on “Hidden Brain”: “testicles aren’t what you think they are. Testicles are really….!!”

  11. Andrew (not the authort) says:

    Wow, this post and the linked article are excellent. The linked article is the best thing I’ve read this week. I’d really love to see a response from Walker.

  12. Terry says:

    So how should Walker respond?

    Walker can simply condescend/sneer/ignore/delay and console himself with his greenbacks (a common and reasonable successful response).

    But is there an honest and reasonably effective response?

    There might be. Walker could start by cataloging all the things that Guzey did not refute. Especially important is to defend the idea that sleep is important. This strategy (tacitly at least) argues that “yeah, sure, maybe there are some weak spots here, but this is important stuff, and don’t throw the baby out with the bathwater.” The objective is to make Guzey sound like a killjoy, opportunistic nitpicker.

    This response might even be an honestly legitimate response. Guzey makes it sound like Walker’s book is overwhelmingly bogus. But is it? Or do Guzey’s criticques apply only to a small fraction of Walker’s points? Guzey’s criticisms are devastating and he there are clearly more than a few serious mistatements. But Guzey doesn’t answer the question of what fraction of Walker’s book is bogus. If Walker is interested in making a serious response, I think that is where he should look.

    • Andrew says:

      Terry:

      One possibility is that Guzey’s criticisms are way off: for example, if Guzey is actually misquoting from Walker’s book or the other sources. In that case, it shouldn’t be hard for Walker to demonstrate this.

      Another possibility is that Guzey’s criticisms are completely accurate, in which case Walker could thank Guzey profusely for pointing out the errors, broadcast the corrections, and as soon as possible issue a corrected version of his book.

      Regarding your suggestion: Sure, that’s fine, but even if the errors are minor, they should be fixed. If Raghuram Rajan didn’t correct the errors in his book that I noted here (and shared with his publisher before the book came out), I’d be annoyed. One of the two errors I noticed was on the very first page of the book!

  13. Mikhail Shubin says:

    I feel proud I was reading this article at 4:00 at night.

  14. Anonymous says:

    And now of course it is in the New York Times, meaning it cannot be dislodged from the upper middle class canon.

    ‘ Matthew Walker, a neuroscientist at the University of California, Berkeley, has this expression, “The shorter you sleep, the shorter your life,” so for the past year and a half I’ve been trying to get better about allocating enough sleep ‘.

    Tech column, Nov. 20, 2019, 9:00 a.m. ET

  15. Kyle C says:

    Walker’s work is now canon, having been referenced in a New York Times tech column. Too late.

    TECH WE’RE USING

    Want the Greenest Device? You May Already Own It

    Nov. 20, 2019, 9:00 a.m. ET

  16. Steve Sailer says:

    As a blogger, I’ve noticed there is a fairly high positive correlation between the number of hours of sleep I get a (plus amount of coffee drunk) and the number of posts I make the following day: 9 hours of sleep and two large cups of coffee ~ 7 posts. 10 hours of sleep and 3 cups of coffee ~ 9 posts.

  17. Jordan Anaya says:

    If you want to see an accusation of misconduct followed by a rebuttal and a rebuttal to that rebuttal you can check out this exchange: https://liorpachter.wordpress.com/2017/08/02/how-not-to-perform-a-differential-expression-analysis-or-science/

Leave a Reply